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INTRODUCTION 


The  increasing  contribution  of  software  development 
and  maintenance  costs  to  the  overall  life-cycle  cost  of  DoD 
weapon  systems  has  been  well  documented  in  recent  years.  In 
particular,  software  life-cycle  costs  are  predicted  to  be  in 
excess  of  80%  of  total  computer  hardware/software  system  life¬ 
cycle  cost  and  in  excess  of  50%  of  the  total  system  costs  by 
the  end  of  the  decade  (Ref.  1).  This  situation  reflects  the 
decreasing  costs  of  computer  hardware  and  the  expanded  use  of 
embedded  computers  in  DoD  systems  because  of  their  functional 
capability.  Recognition  of  this  accelerating  shift  of  cost 
drivers  from  hardware  to  software  has  resulted  in  s i gn i f i cant  1 y 
more  attention  being  given  to  methods  of  deriving  estimates  of 
the  resources  that  will  be  expended  on  the  software  subsystems. 
Therefore,  a  method  of  estimating  software  cost_  and  schedule 
requirements  with  a  reasonable  con f i dence  level  is  required  to 
enhanc  e  ex  i  s  t  i  rig  manageme n  t  t  oo  1  s  . 

There  currently  exist  several  software  cost  estimat¬ 
ing  systems  and  models  which  have  gained  some  degree  of  accep¬ 
tance  within  the  software  cost  estimating  community.  The  most 
commonly  used  of  these  are  S1.1M  (Ref.  2),  JS-1  (Ref.  '{  )  ,  RRICK-S 
(Ref.  A),  and  C'.OCOMO  (Ref.  5).  However,  before  any  of  these 
models  can  be  used  to  develop  a  software  cost  estimate,  they 
should  be  validated  for  use  for  a  particular  class  of  applica¬ 
tions.  Moreover ,  if  validated,  these  models  must  then  be  cal¬ 
ibrated  for  a  particular  development  environment.  A  data  base 
which  is  representative  of  Air  Force  Electronic  Systems  Division 
(F.SD)  software  development  is  necessary  to  make  these  models 
useful . 
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An  al  ternat ive  Lo  the  use  of  the  existing  software 
cost  models  is  the  development  of  a  unique  model  which  is  spe¬ 
cifically  tailored  to  the  ESD  development  environment  and  pro¬ 
duct  characteristics.  This  approach  can  overcome  several 
shortcomings  of  the  existing  models,  in  particular,  the  lack 
of  a : 

•  Valid  method  for  estimating  the  cost  of 
software  modification  or  maintenance 

•  Valid  technique  for  timephasing  manpower 
requirements  when  resource  constraints 
exist 

•  Statistical  basis  for  establishing  confi¬ 
dence  intervals  for  an  estimate 

•  Capability  to  estimate  software  size. 

Development  of  this  unique  model  requires  a  comprehensive  data 
base  which  contains  software  characteristics  and  parameters 
for  development  projects  that  reflect  the  development  environ¬ 
ment  and  applications  of  ESD. 


1 . 1  PURPOSE 

The  main  objective  of  this  effort  i  s  _t_o  develop  a 

so  ft  wa  re  c  <  >sj  da  t  _a  base  which  can  s_uppo  r  t t  he  oo  s  t  es  t  i  mat  - 

ing  process  .  The  ability  to  use  the  data  base  for  validation 
and  calibration  of  the  COCOMO ,  SLIM,  JS-1,  and  PR1CE-S  soft¬ 
ware  cost  models  is  a  major  consideration.  Since  the  state- 
of-the-art  of  software  cost  estimating  is  advancing,  the  data 
base  must  also  be  flexible  enough  to  be  used  for  model  devel¬ 
opment  or  enhancement.  Finally,  the  data  base  should  support 
development  of  software  sizing  tools  because  size  is  the  key 
input  to  most  software  models.  Figure  1.1-1  depicts  a  multi¬ 
purpose  software  cost  data  base. 
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Figure  1.1-1  Software  Cost  Data  Base  Implementation 

In  addition  to  defining  a  software  cost  data  base, 
this  effort  includes  the  design  of  data  collection  formats, 
the  development  and  implementation  of  an  approach  for  the  ini¬ 
tial  data  collection,  and  the  est at : i 'hment  of  a  methodology 
for  maintenance  and  growth  of  the  data  o  se  in  the  future. 
Recommendations  for  future  uses  of  the  data  ba;jn  and  approach*1 
for  advancing  the  state-of-the-art  of  software  cost  estimation 
are  also  part  of  this  effort. 


1 . 2  GENERAL  APPROACH 

The  data  base  design  is  the  result  of  discussions 
with  government  and  industry  personnel  involved  in  the  soft¬ 
ware  development  process  as  either  software  engineers,  cost 
estimators  or  data  base  developers.  It  was  further  refined  by 
using  information  from  surveys  of  reports  on  previous  data 
collection  and  data  base  development  efforts,  by  review  of  the 
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user  documentation  for  numerous  software  cost  estimating  models, 
and  by  analysis  of  prior  software  productivity,  quality  and 
reliability  metrics  research. 

A  standard  software  work  breakdown  structure  (WBS) 
was  defined  and  is  used  for  the  data  base  structure.  The 
selected  structure  is  based  on  an  analysis  of  historical  WBSs 
used  at  ESD,  other  structures  proposed  by  industry  and  a  com¬ 
parison  with  WBS  practices  for  hardware. 

Based  on  the  results  of  this  research,  data  collec¬ 
tion  formats  were  then  developed  and  distributed  to  potential 
participants.  Analysis  of  the  feedback  from  the  participants 
and  the  completed  data  collection  formats  were  used  to  refine 
the  data  collection  package. 

Finally,  the  software  cost  estimation  requirements  of 
ESD  were  analyzed  to  determine  what  enhancements  were  needed  to 
improve  the  current  capability. 


1.3  REPORT  ORGANIZATION 

This  report  consists  of  two  volumes:  Volume  I  -  a 
report  on  the  data  base  designs  and  data  collection  methodology; 
Volume  II  -  a  compilation  of  the  data  collected.  The  following 
describes  the  contents  of  Volume  I. 

Chapter  2  of  this  report  details  the  research  performed 
to  determine  the  data  base  contents  and  describes  the  elements 
that  are  included,  as  well  as  the  structure  of  the  data  base. 

In  Chapter  3  the  development  and  implementation  of  the  data  col¬ 
lection  formats  and  methodology  are  discussed  and  evaluated; 


recommendations  for  growth  of  the  data  base  are  also  made.  Chap¬ 
ter  A  details  future  efforts  that  should  be  undertaken  to  con¬ 
tinue  improving  the  state-of-the-art  of  software  cost  estimation. 

A  summary  of  the  study  results  and  recommendations  is  presented 
in  Chapter  5.  Appendix  A  contains  cross  reference  tables  be¬ 
tween  the  cost  model  input  parameters  and  data  collection  formats. 
The  final  version  of  the  data  collection  package  is  contained 
in  Appendix  B. 
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2 .  DATA  BASE  REQUIREMENTS  ANALYSIS 

The  software  development  process  represents  the  com¬ 
plex  interaction  of  requirements  and  resources  to  produce  a 
software  product.  Figure  2-1  depicts  some  of  the  major  cate¬ 
gories  of  factors  which  affect  the  outcome  of  this  process. 

The  software  data  base  must  contain  the  data  elements  neces¬ 
sary  to  measure  and  evaluate  the  impact  of  these  factors. 

The  contents  and  structure  of  the  data  base  were  deter¬ 
mined  through  an  analysis  of  the  data  reporting  practices  used 
and  the  data  bases  maintained  by  organ i za t i ons  within  government 
and  industry.  Of  particular  interest  were  those  software  data 
bases  whose  objective  was  to  support  cost  estimating  or  to 
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evaluate  software  productivity.  Likewise,  the  data  require¬ 
ments  for  use  and  calibration  of  the  major  software  cost  models 
in  use  today  were  reviewed.  Finally,  additional  research  was 
performed  in  the  area  of  software  productivity  and  quality 
metrics  to  evaluate  alternatives  to  the  current  methods  of 
software  cost  estimation. 

2.1  EXISTING  DATA  BASES 

Currently,  software  data  bases  fall  into  two  cate¬ 
gories:  those  that  contain  summary  daLa  at  the  system  level 

and  those  that  contain  detail  data  at  the  lowest  level  to  which 
software  can  be  logically  subdivided.  To  a  great  extent  the 
amount  of  detail  is  determined  not  by  the  requirements  of  the 
data  base  developer,  but  by  the  availability  of  the  data.  The 
most  detailed  data  bases  exist  within  the  development  organi¬ 
zations  which  have  a  direct  influence  on  the  type  of  data  col¬ 
lected  for  the  day-to-day  management  of  a  development  effort. 
Therefore,  government  software  development  organizations,  such 
as  the  NASA  Software  Engineering  Laboratory,  and  defense  con¬ 
tractors  historically  have  the  best  data  available.  On  the 
other  hand,  data  availability  at  government  program  offices  is 
limited  by  the  existing  data  items  used  for  reporting  software 
technical  and  resource  utilization  data. 

Over  the  past  few  years  the  Data  and  Analysis  Center 
tor  Software  ( DACS ) ,  the  National  Security  Agency  (NSA),  and 
ESD ,  among  others,  have  endeavored  to  develop  new  data  items 
and  reporting  methods  to  obtain  software  data  with  sufficient 
detail  to  support  the  cost  estimating  process  and  to  develop 
better  estimating  tools.  The  ESD  efforts  have  resulted  in  the 
development  of  the  Software  Acquisition  Resource  Expenditure 
(SARK)  Data  Collection  Methodology,  which  was  completed  in 


December  1983  (Ref.  6).  Both  DACS  and  NSA  are  evaluating  SAKE 
to  determine  its  applicability  to  their  data  needs. 


In  order  to  capitalize  on  the  effort  expended  in  the 
development  of  SARE ,  this  project  used  SARE  as  the  starting 
point  for  the  data  base  requirements  analysis.  Additions  have 
been  made  to  the  specific  elements  collected  to  provide  infor¬ 
mation  for  definitive  c  lass i f ica t ion  of  the  system  and  its 
Computer  Program  Configuration  Items  (CPCIs)  and  to  provide 
cross  checks  for  other  data  entries.  On  the  other  hand,  some 
of  the  detail  required  by  SARE  has  been  eliminated  because  it 
is  not  practical  to  collect.  For  example,  no  data  are  collected 
at  the  Computer  Program  Component  level  with  the  exception  of 
function  and  sizing  data  if  that  is  the  lowest  level  at  which 
these  data  are  available.  Add i t iona 1 1 y ,  parameters  which  cannot 
be  reasonably  determined  outside  of  the  development  organization 
at  the  time  when  an  estimate  is  being  prepared  were  also  deleted. 

2.2  SOFTWARE  COST  MODEL  DATA  REQUIREMENTS 

A  thorough  review  of  the  documentation  for  the  major 
software  cost  models  in  use  today  (Refs.  2  through  5  and  7 
through  14)  resulted  in  the  tabulation  of  the  common  elements 
among  them.  Appendix  A  contains  cross  reference  tables  showing 
the  relationship  between  the  data  requirements  for  the  COCOMO, 
JS-1,  SLIM,  and  PR1CE-S  models  and  the  data  items  on  the  data 
collection  formats.  For  those  items  which  were  subjective  in 
nature,  the  guidelines  for  making  the  value  judgement  were 
reviewed  to  identify  any  objective  factors  or  characteristics 
that  could  be  used  to  make  the  proper  determination.  This  was 
not  feasible  in  all  cases,  e.g.,  the  complexity  factor  used  by 
COCOMO  has  extensive  guidelines  which  would  require  an  inordi¬ 
nate  number  of  numerical  statistics  to  replace  the  subjective 


determination  which  would  be  made  by  a  software  engineer  famil¬ 
iar  with  the  project.  Whenever  there  was  an  overlap  among  the 
models,  data  representing  the  lowest  coinmon  denominator  and 
with  the  greatest  level  of  granularity  were  included  in  the 
data  base. 

In  areas  where  ambiguity  may  exist  even  with  exten¬ 
sive  data  entry  guidelines,  data  elements  on  which  the  ambig¬ 
uous  parameter  would  be  based  were  also  included  in  the  data 
base  to  provide  consistency  checks.  For  example,  an  assessment 
of  the  CPU  memory  constraint  as  a  percent  utilization  of  the 
total  available  memory  does  not  necessarily  indicate  a  true 
memory  constraint.  Therefore,  information  about  the  expansion 
capability  of  the  target  computer  and  the  reserve  memory  re¬ 
quirements  is  also  included. 

2.3  SOFTWARE  PRODUCTIVITY  AND  QUALITY  METRICS 

The  use  of  lines  of  code  as  a  measurement  of  software 
productivity  yields  a  wide  range  of  results  depending  on  the 
project.  Although  alternatives  to  lines  of  code  have  been 
proposed  over  the  years,  it  appears  that  lines  of  code  will 
remain  the  standard  for  cost  estimation  purposes.  Many  of  the 
alternative  approaches  (Refs.  15  through  2U )  require  the  meas¬ 
urement  of  parameters  that  can  only  be  determined  after  the 
completion  of  the  software  development  effort,  such  as  weighted 
statement  count  and  process  (a  measure  of  the  number  of  data 
items  and  paths  in  a  program).  Still  others  are  highly  cor¬ 
related  to  lines  of  code ,  for  example,  number  of  modules.  In 
general,  the  research  performed  on  software  productivity  meas¬ 
urement  indicates  that,  for  cost  estimation,  lines  of  code  is 
the  most  promising  metric  when  used  in  combination  with  quali- 
t  a  t  i ve  info  rma  t i on . 
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Therefore,  when  lines  of  delivered  source  code  are 
used  as  the  primary  metric,  the  variation  in  productivity  from 
one  project  to  another  must  be  accounted  for  by  qualifying  the 
amount  of  code  written  using  additional  productivity  metrics 
and  by  the  addition  of  quality  metrics.  Many  productivity 
metrics  are  nonquan t i ta t i ve ,  such  as  functions  performed,  devel¬ 
opment  constraints,  and  personnel  quality.  These  can  be  used 
strictly  as  qualitative  metrics  for  categorization  of  develop¬ 
ment  projects  into  homogeneous  groupings.  Alternatively,  they 
can  be  quantified  using  a  relative  scale,  as  has  been  done  in 
the  COCOMO  and  JS-1  models,  with  detailed  qualitative  guide¬ 
lines  for  selecting  a  numeric  value.  The  same  factors  apply 
to  quality  metrics,  such  as  reliability,  maintainability,  or 
completeness  . 

Both  approaches  are  valid  and  should  be  used  in  con¬ 
junction  with  one  another.  To  the  extent  that  a  sufficiently 
large  data  base  exists,  the  available  estimation  models  should 
be  calibrated  using  the  most  narrowly  defined  grouping  of  proj¬ 
ects  possible.  The  development  of  new  models  should  also  be 
based  on  the  most  homogeneous  grouping  of  projects  that  can  be 
selected  through  the  use  of  productivity  and  quality  metrics 
and  still  contain  sufficient  data  points  to  be  statistically 
valid. 

This  data  base  is  designed  to  include  the  metrics 
themselves,  based  on  a  definitive  set  of  guidelines,  or  the 
lower  level  data  elements  required  to  derive  the  metrics. 


2.4  DATA  BASE  CONTENTS 


The  data  base  can  be  logically  divided  into  six  dis¬ 
tinct  categories: 


\-  v'  ■ 


•  System  description  and  characteristics 

•  Development  schedule  data 

•  Hardware  characteristics  and  constraints 

•  Development  resources  and  constraints 

•  Software  size  and  cha rac t eri s t i cs 

•  Resource  expenditure  data. 

The  data  within  each  of  these  categories  consist  of  the  ele¬ 
ments  required  to  classify  the  system,  to  define  the  develop¬ 
ment  environment,  and  to  derive  the  software  development  cost 
drivers  and  input  parameters  for  software  cost  and  sizing  model 

2.4.1  System  Description  and  Characteri st i cs 

A  key  element  of  any  data  base  design  is  the  develop¬ 
ment  of  a  c lass i f i ca t ion  scheme  that  can  be  used  to  group  data 
for  analysis  and  for  data  retrieval.  The  development  of  a 
comprehensive  list  of  keywords  based  on  the  descriptions  of  a 
weapon  system  at  the  total  system  level  is  essential  for  match¬ 
ing  a  new  system  against  those  historical  data  which  are  most 
appropriate  for  model  calibration.  By  extending  this  technique 
to  the  software  segment  of  the  system,  to  the  CPCI  level,  and 
on  down  to  the  module  level,  a  more  restrictive  selection  can 
be  made  as  the  new  system  is  defined  in  greater  detail.  Fig¬ 
ure  2.4-1  shows  the  elements  used  for  this  progressive  classifi 
cation  of  a  system. 

Descriptive  data  defining  the  mission  of  the  system, 
the  hardware  interfaces,  the  functions  performed  by  the  system 
as  a  whole  and  by  the  lower  level  components  of  the  system  are 
included  in  this  data  base  so  that  keywords  for  data  retrieval 
and  classification  can  be  developed.  As  the  number  of  projects 
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SOFTWARE  DEVELOPMENT  PROJECT 


•  MISSION  DESCRIPTION 

•  SYSTEM  FUNCTIONS 

•  TARGET  COMPUTER  CHARACTERISTICS 


•  HARDWARE  INTERFACES 

•  SOFTWARE  FUNCTIONS 


COMPUTER  PROGRAM 

•  FUNCTIONAL  DESCRIPTION 

•  SIZE  PROFILE  BY  OPERATION 

•  SOURCE  STATEMENT  TYPE  MIX 

•  PROGRAMMING  LANGUAGE 


CONFIGURATION  ITEM 

•  DEVELOPMENT  PERSONNEL  QUALITY 

•  OPERATIONAL  RESPONSE  REQUIREMENT 

•  DISPLAY  REQUIREMENT 

•  TARGET  COMPUTER  CHARACTER  I STI CS 


MODULE  OK  UNIT 


•  FUNCTIONAL  CATEGORY 


•  PROGRAMMING  LANGUAGE 


Figure  2.4-1  Software  Classification  Hierarchy 

documented  in  the  data  base  increases,  some  of  the  other  data 
categories,  such  as  hardware  constraints,  can  also  be  used  to 
further  restrict  data  selection  or  to  provide  a  more  homo¬ 
geneous  grouping. 

2.4.2  bey e  1  <  rpmen  t  Schedu  1  e  I)a  t  a 


Schedule  data  are  collected  by  major  development  milt 
stone  at  the  svstem  level  arid  tor  each  (PCI  in  the  system. 
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These  mi les tones  define  the  schedule  to  a  1 evel  of  detail  suf¬ 
ficient  to  segment  the  effort  into  the  major  phases  of  the 
software  life  cycle,  from  project  start  to  the  start  of  the 
maintenance  phase.  These  data,  in  conjunction  with  the  monthly 
resource  expenditure  data,  are  used  to  determine  the  relative 
effort  expended  for  each  phase  of  the  development.  Both  the 
original  schedule  dates,  as  well  as  the  actual  or  estimated 
milestone  completion  dates,  are  included  so  that  an  assessment 
can  be  made  of  the  degree  of  schedule  acce 1 erat i on .  A  subjec¬ 
tive  rating  of  the  perceived  schedule  acceleration  or  stretchout 
is  also  included  to  determine  its  impact  on  the  development. 

2.4.3  Hardware  Characteristics  and  Constraints 

Information  about  the  target  computer  is  necessary  to 
determine  the  timing  and  sizing  constraints  under  which  the 
software  is  being  developed.  Central  processing  unit  (CPU) 
memory  and  timing  utilization  data  are  specified  with  supple¬ 
mental  information  indicating  the  use  of  ex t raord i nary  meas¬ 
ures  to  reduce  size  and  the  amount  of  software  which  is  time 
constrained.  Additional  data  about  the  expansion  capability 
of  the  hardware,  the  reserve  memory  and  time  requirements  are 
also  included  to  determine  whether  the  constraints  indicated 
are  real  or  perceived. 

To  allow  for  the  grouping  of  software  developments  by 
class  of  target  computer,  especially  for  standard  architectures, 
the  computer  used  is  specifically  identified  and  the  maturity 
of  the  hardware  and  the  virtual  machine  is  assessed.  Instances 
of  concurrent  hardware  and  software  development  are  indicated. 


2 . A . 4  Development  Resources  and  Constraints 

There  is  a  general  consensus  that  the  development  en¬ 
vironment  and  personnel  factors  are  major  contributors  to  the 
variation  in  productivity  among  software  developments.  This 
data  base  is  designed  to  characterize  the  major  components  of 
the  development  environment  which  impact  software  cost. 

The  tools  and  methods  available  within  the  develop¬ 
ment  organization  are  identified  at  the  system  level  and  their 
usage  is  rated  for  each  CPCI .  The  characteristics  of  the  devel 
opment  computer  are  compared  with  those  of  the  target  computer. 
Since  access  to  the  computer  resource  has  a  direct  impact  on 
productivity,  the  availability  of  the  computer,  as  well  as  the 
access  mode  and  computer  location,  are  specified. 

In  the  area  of  hardware  cost  estimation,  it  has  been 
accepted  practice  to  apply  an  improvement  curve  to  the  produc¬ 
tion  of  multiple  units  of  an  item.  Although  a  direct  analogy 
cannot  be  made  for  applying  an  improvement  curve  to  software 
development,  the  experience  of  the  development  personnel  in 
the  weapon  system  application  and  with  the  development  tools 
and  techniques  used  will  impact  the  overall  productivity  on  a 
project.  While  software  developers  are  involved  in  the  famil¬ 
iarization  process  with  the  development  environment,  the  design 
and  code  produced  are  often  less  than  optimal  and  result  in 
increased  failure  rates  and  redesign  effort .  Therefore,  infor¬ 
mation  about  the  experience  of  the  personnel  assigned  at  the 
start  of  the  project  is  captured  in  the  data  base. 


In  addition  to  the  effects  of  personnel  quality  on 
the  costs  of  a  project,  the  availability  of  personnel  will 
likewise  affect  the  schedule  and  man  loading  profiles  for  .1 
project.  Although  these  data  are  not  a  direct  input  into  any 


of  the  models,  they  can  be  used  to  analyze  calibration  results, 
in  particular  for  SLIM  which  uses  project  duration,  as  well  as 
development  manpower,  in  the  calibration  process  with  the 
assumption  that  manloading  is  uncons t ra i ned . 

2.4.5  Software  Size  and  Char  act  eri  st  i_c  s 

The  type  of  software  size  data  included  in  the  data 
base  is  driven  by  two  requirements: 

•  The  need  for  size  data  at  the  CPC1  level 
with  allocations  to  various  functional 
characteristics,  processing  modes,  and 
languages  to  support  the  specific  re¬ 
quirements  of  several  cost  models 

•  The  need  for  size  decomposition  to  the 
lowest  level  available  with  functional 
cat egor i za t  1  on  and  language  identifica¬ 
tion  to  support  sizing  by  analogy 

requ i remen  t s . 

The  mode  1  -  spec i f i c  allocation  data  at  the  CRCl  level,  such  as 
source  statement  mix,  can  also  tie  used  to  group  CPCIs  into 
homogeneous  groupings  for  model  calibration  and  model  develop¬ 
ment  research.  The  lower  level  of  functional  detail  provides 
data  for  in-depth  analysis  of  variations  in  productivity  or 
calibration  results  among  a  group  of  programs  that  appear  to 
be  homogeneous  at  the  CPU  I  level. 

To  further  clarify  the  magnitude  of  the  development 
task,  additional  data  about  the  amount  of  reuseable  or  modified 
code  is  required.  This  element  will  be  particularly  useful 
for  evaluating  the  impact  that  Ada  will  have  in  this  area. 

2 . 4  .  Resource  Expenditure  Data 

Ordinarily,  cost  data  are  obtained  in  terms  of  dollars 
However,  there  are  many  problems  inherent  in  that  approach. 


First  of  all,  the  data  must  be  normalized  to  a  common  base 
year  to  eliminate  the  distortion  which  is  caused  by  inflation 
and  to  make  data  from  different  time  frames  comparable.  Next, 
the  effects  of  different  labor  rates  due  to  the  geographic 
locations  of  the  developers  must  be  accounted  for  in  the  data. 
Finally,  the  data  must  be  normalized  for  the  impact  of  differ¬ 
ent  overhead  rates  and  charging  practices  for  other  direct 
costs.  This  normalization  process  is  very  intricate  and  would 
require  the  collection  of  additional  data  elements,  such  as 
labor  and  overlie. id  rates.  In  many  instances,  t  hese  data  are 
proprietary  to  a  particular  company  and  difficult  to  obtain 
because  they  are  competitive  sensitive. 

Most  data  norma  1 i za t  1  on  problems  are  avoided  when  the 
cost  data  are  collected  in  terms  of  manpower.  The  only  adjust¬ 
ment  required  in  this  case  is  the  conversion  of  the*  data  to  a 
common  unit  of  manpower  measurement  ,  such  as  manmonths  composed 
of  a  specific  number  of  manhours.  The  labor  content  of  t he 
manpower  is  evaluated  to  insure  that  the-  data  for  all  projects 
represent  the  same  types  of  activities,  for  example,  designer1 
and  programmers,  but  not  clerical  support  . 


The  F.SD  data  base  design  requires  that  resource  utili¬ 
zation  data  be  expressed  in  cciuivah-nl  manmonths  (17b  hours)  an 
timephased  in  terms  of  months  a f t e r  contract  award  or  project 
start.  I  he  data  are  timephased  so  that  they  c  a  1 1  be  used  l  n 
e\  aluate  the  Rayleigh  curve  1  imephasing  used  bv  SI.  1 M  and  JS-1 
and  the  resource  al  local  ion  by  phase  method  in  COCOMO .  They  c  a 
also  he  used  to  select  an  appropriate  method  from  t  fie  choices 
•  ■  1  f  1  ra-d  in  1’K  ]  C.’F  -  S  . 


2.5 


WORK  BREAKDOWN  STRUCTURE 


Since  the  work  breakdown  structure  (WBS)  is  the  scheme 
generally  used  for  organizing  cost  data  and  for  reporting  cost 
performance  on  DoD  programs,  the  WBS  methodology  has  been  se¬ 
lected  as  the  structure  for  organizing  the  software  cost  data 
base.  This  decision  required  the  development  of  a  standard 
software  work  breakdown  structure  for  integration  into  a  project 
WBS  defined  in  accordance  with  MIL-STD-881A  (Ref.  25).  In  order 
to  facilitate  comparison  of  projects  at  the  lower  levels  of  de¬ 
tail,  ESD  must  adopt  a  standard  WBS  with  detailed  definitions 
comparable  to  those  in  M I L-STD-881A .  Ideally,  this  standard 

would  be  accepted  by  the  other  DoD  software  developers. 

The  WBS  definition  began  with  a  review  of  the  WBSs  in 
use  for  cost  performance  data  reporting  at  ESD.  The  analysis 
encompassed  all  active  ESD  programs,  as  well  as  projects  com¬ 
pleted  within  the  last  ten  years.  In  general,  older  projects 
did  not  use  a  detailed  WBS  for  software  and  in  many  cases  the 
software  effort  was  located  below  the  reporting  level.  Over 
the  years,  the  level  of  software  detail  reported  has  been  in¬ 
creasing  in  parallel  with  the  growth  in  the  importance  of  soft¬ 
ware  for  ESD  systems.  The  majority  of  projects  now  obtain 
cost  data  at  the  CPC l  level. 

The  results  of  the  above  review  were  then  compared 
with  several  proposed  software  work  breakdown  structures  (Rets. 
2b,  27,  28).  In  contrast  to  the  product -oriented  structure 

ident  if  ied  in  M  I  1.- STD- 88  1  A  ,  many  of  the  proposed  approaches 
have  either  an  accounting  or  a  functional  orientation.  However, 
these-  other  approaches  are  not  incompatible  with  a  product - 
oriented  WBS,  since  the  functional  and  accounting  shredouts 
can  be  made  within  a  product -oriented  WBS  below  the  lowest 
product  level  of  interest  .  For  software  developments  the  CPC1 
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level  most  commonly  used  for  cost  estimating.  Breakouts  below 
this  level  are  either  artificial,  such  as  the  CPC,  or  are  at 
too  great  a  level  of  detail  for  cost  effective  data  collection, 
such  as  the  module.  Therefore,  the  CPCI  has  been  selected  as 
the  key  element  to  be  included  in  the  software  work  breakdown 
structure.  Extension  of  the  WBS  below  this  level  will  be  left 
to  the  discretion  of  the  developer. 


Above  the  CPCI  level,  a  software  effort  can  be  log¬ 
ically  subdivided  at  the  system  level  into  one  or  more  sub¬ 
systems.  Each  subsystem  can  then  be  divided  into  one  or  more 
CPCls.  Although  a  software  subsystem  is  a  logical  and  not  a 
physical  entity,  it  is  still  analogous  to  a  hardware  subsystem 
and  should  be  treated  in  the  same  manner.  The  recommended  WBS 
shown  in  Table  2.5-1  is  an  enhancement  of  a  MIL-STD-881A  WBS 
with  software  subsystems  in  parallel  with  hardware  subsystems 
at  level  three.  Support  software  is  treated  as  a  separate  soft 
ware  subsystem.  Ea,  h  software  subsystem  is  extended  to  level 
four  into  a  breakout  of  the  subsystem  design  activity,  the 
subsystem  integration,  and  the  CPCls  of  which  it  is  composed. 
This  WBS  is  based  on  an  alternative  presented  in  SAKE.  The 
other  alternatives  in  SAKE  were  rejected  because  they  do  not 
cover  any  situations  that  cannot  be  handled  within  the  recom¬ 
mended  approach  and  could  result  in  confusion  and  inconsistent 
app 1 i c  a  t i on . 


During  the  data  collection  effort  the  recommended  WBS 
was  discussed  with  several  defense  contractors  and  the  general 
reaction  was  favorable.  The  recommended  WBS  was  similar  to 
those  being  adopted  by  these  contractors  for  their  internal 
management  requirements.  These  cont rac tors  also  stressed  the 
need  for  standardization. 
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TABLE  2.5-1 

DEFENSE  SYSTEM  WORK  BREAKDOWN  STRUCTURE 


Level  1 


Level  2 


Level  3 


Level  4 


Defense  System 


Prime  Mission  Equipment 

Integration  and  Assembly 
Hardware  Subsystem  or  End  Item  1 


Hardware  Subsystem  or  End  Item  n 
•'^Software  Subsystem  1 

Subsystem  Analysis  &  Design 
Subsystem  Integration  &  Test 
Computer  Program  Configuration  Item  1 


Computer  Program  Configuration  Item  n 


'-’-Software  Subsystem  n 
•'■'Support  Software 

Computer  Program  Configuration  Item  1 


Computer  Program  Configuration  Item  n 

Training 

Equipment 
Se  rv i ces 
Facilities 

Peculiar  Support  Equipment 

Organ i zat l ona 1 
I nte  rmed i a  te 
Depot 

System  Test  &  Evaluation 

Development  Test  &  Evaluation 
Operational  Test  &  Evaluation 
Mockups 

Test  &  Evaluation  Support 
Test  Facilities 
Sys tem/ Program  Management 

Systems  Engineering 
Project  Management 


•Replaces  Computer  Program  at  Level 
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TABLE  2.5- 


DEFENSE  SYSTEM  WORK  BREAKDO 


Level  l 


Leve  1  2 


Leve  1  3 


Level  4 


Technical  Publications 
Engineering  Data 
Management  Data 
Support  Data 
Data  Depository 
Operat iona 1 /Si te  Activation 

Contractor  Technical  Support 
Site 

Construct i on 

Site/Ship/Vehicle  Conversion 
System  Assembly  Installation  & 

Checkout  on  Site 
Common  Support  Equipment 

Organ i zat i ona 1 
1 ntermediate 
Depot 

Industrial  Facilities 

Const  ruction /Con version /Expans  ion 
Equipment  Acquisition  or  Modernization 
Ma  i  iitenarice 

Initial  Spares  &  Initial  Repair  Parts 


2.6  DATA  BASE  STRUCTURE 

The  data  babe  is  organ i zed  using  the  recommended  WHS. 

At  level  one,  the  defense  system,  data  describing  the  system 
mission,  major  functions,  hardware  interfaces,  development 
tools  and  methodologies,  system  level  documentation  page  counts, 
rind  change  history  are  collected.  Additional  data  describing 
product  characteristics,  the  development  environment,  and  devel¬ 
opment  resources  are  collected  at  the  CPC1  level  for  t  host' 
elements  th.it  are  different  lor  each  CPC  I.  Schedule  and  fail¬ 
ure  data  are  also  collected  .it  this  level. 

Sizing  and  functional  data  are  collected,  at  a  minimum 
to  the  CPU  I  level  for  cost  est  imat  ion  and  down  to  the  lowest 
level  available  for  size  estimation.  Resource  expenditure 
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data  are  collected  for  every  element  of  the  WBS  that  applies 
to  software  only.  For  projects  that  are  entirely  software, 
cost  data  are  collected  for  all  elements  of  the  WBS. 

During  the  data  retrieval  process,  analogies  are  pro¬ 
gressively  developed  starting  at  the  defense  system  level  to 
select  homogeneous  projects  for  further  analysis,  continuing 
at  the  CPCI  level  to  select  CPCIs  for  software  cost  model  vali¬ 
dation/calibration  or  development,  and  at  the  module  level  for 
software  sizing.  Depending  on  the  size  of  the  data  base  and 
the  sample  size  requirements  for  a  statistically  valid  analysis, 
restrictive  criteria  are  selected  from  the  characteristics  and 
functions  at  the  appropriate  level  to  select  a  homogeneous 
subset  of  the  data  base. 


3. 


DATA  COLLECTION  METHODOLOGY 


The  establishment  of  a  data  base  from  the  design  de¬ 
scribed  in  Chapter  2  required  the  development  of  data  collec¬ 
tion  formats  and  a  data  collection  methodology.  The  approach 
taken  also  provides  for  use  of  the  formats  for  future  data 
collection  to  maintain  the  data  base  and  for  data  collection 
for  cost  estimating  model  input.  Appendix  A  contains  tables 
which  cross-reference  the  COCOMO ,  SLIM,  PR1CE-S,  and  JS-1  cost 
model  input  parameters  to  the  data  collection  formats.  In 
addition  to  detailing  the  methodology  developed  for  the  initial 
collection,  this  chapter  also  proposes  approaches  for  data 
base  maintenance  and  growth. 


3.1  COLLECTION  FORMATS 

The  data  collection  formats  were  developed  using  t he 
Mitre  SAKE  (Ref.  b)  formats  as  a  strawman.  Formats  developed 
by  DACS ,  NSA,  NASA-SEL,  and  the  Aerospace  Corporation  were  also 
evaluated  and  the  best  elements  were  selected  from  each  of  the 
different  approaches.  Other  elements  represent  new  designs  to 
enhance  clarity  and  useablity  or  to  add  unique  data  items. 
Four  separate  formats  were  originally  developed  to  allow  for 
the  different  software  decomposition  levels  at  which  data  items 
would  be  collected.  As  a  result  of  this  initial  ’data  collec¬ 
tion  effort,  a  fifth  format  was  developed  to  separately  collect 
hardware  data.  The  data  collection  package  also  includes  a 
comprehensive  set  of  instructions.  Figure  3.1-1  summarizes 
the  contents  of  each  element  of  the  data  collection  package. 
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Figure  3.1-1  Data  Collection  Package 
3.1.1  Software  Development  Project  Summary  Data 

This  format  (Figure  3.1-2)  is  designed  to  collect  data 
at  the  top  level  of  the  WBS,  the  defense  system.  It  includes 
descriptions  of  the  mission  of  the  system  as  a  whole,  specifi¬ 
cation  of  the  hardware  interfaces  required,  identification  of 
the  system  functions  and  the  allocation  of  those  functions  to 
the  software  elements  of  the  system.  These  data  are  used  for 
classification  of  the  system  so  that  it  can  be  grouped  with  sim¬ 
ilar  systems  for  analysis  and  for  model  validation  or  calibration 

The  format  also  contains  a  checklist  of  commonly  used 
software  development  tools  and  techniques.  These  checklists 
are  used  in  conjunction  with  data  elements  on  the  CPC  I  Summary 
Data  format  to  develop  inputs  for  several  cost  models  requiring 
an  assessment  of  the  development  environment.  A  schedule  at 
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the  total  project  level  is  included  to  facilitate  allocation 
of  resource  expenditure  data  to  the  different  phases  of  the 
software  development  process. 

System  level  documentation  page  counts  are  requested 
as  a  prospective  variable  for  use  in  estimating  documentation 
cost  and  for  measuring  software  maintainability.  A  software 
change  history  is  also  included  for  an  evaluation  of  software 
requirements  volatility  and  its  impact  on  software  cost. 

A  copy  of  this  format  is  prepared  for  each  project 
and  is  used  both  for  additions  to  the  data  base  and  to  collect 
cost  model  input  parameters  for  an  estimate.  It  can  also  be 
used  to  collect  data  for  analogy  selection  at  the  project  level 

3.1.2  Comput  er  Hardware  be t  a i 1  Da t  a 

This  format  (Figure  3.1-3)  is  designed  to  collect  data 
about  the  target  computer  on  which  the  software  product  will 
operate  and  about  the  computer  on  which  the  software  is  being 
developed.  The  constraints  of  the  target  computer  are  identi¬ 
fied  to  define  the  operational  environment  for  the  software. 
The  memory  and  time  utilization  by  the  software  is  specified, 
along  with  reserve  requirements  and  expansion  capability,  so 
that  an  assessment  can  be  made  of  constraints  on  the  develop¬ 
ment  process  caused  by  these  operational  requirements.  The 
development  computer  characteristics  and  its  effectiveness  as 
a  resource  are  likewise  addressed. 

This  format  is  prepared  for  each  target  computer  in 
the  system  and  its  related  development  computer.  It  is  used 
for  additions  to  the  data  base  and  for  collection  of  cost  model 
i  n p u  t  pa.  rame  tors. 


Figure  3 . 1  -  3  Development  and  Target  Computer  Data  Form 
3.1.3  CPC  I  Summary  Data 

This  format  (Figure  3.1-4)  is  designed  to  collect  data 
at  the  CPC!  level.  The  data  required  at  this  level  are  deter¬ 
mined  primarily  by  the  input  parameters  of  the  COCOMO,  SLIM, 
PRICF.-S,  and  JS-1  cost  models. 

The  format  begins  with  the  name  of  the  CPC  l  and  a 
narrative  description  which  can  be  used  for  analogy  selection 
and  homogeneous  grouping  <>f  CPCls.  Next  ,  the  experience  of 
t  Fie  development  personnel  t'elat  ive  to  the  development  environ¬ 
ment  and  the  software  application,  as  well  as  the  quality  of 
the  personnel ,  is  assessed. 

1  he  balance  of  the  format  is  concerned  with  the  charm 
teristics  and  size  of  the  software.  As  required  by  several  of 


the  cost  models,  the  software  size  is  allocated  to  different 
functional  operations,  source  statement  mixes,  programming 
languages,  and  operational  modes.  These  size  profiles  can  also 
be  used  for  developing  analogies.  Several  software  quality 
measures  required  by  the  models  are  included.  Additionally,  two 
prospective  measures  of  software  quality,  that  is,  documentation 
page  count  and  ruilure  history,  are  collected  for  future  research 

This  format  is  prepared  for  each  CPC  I  in  the  system  and 
can  be  used  for  additions  to  the  data  base  or  to  collect  cost 
model  input  parameters. 

1 . 1 . h  Resource  Expend i t  urp  Data 


This  format  (Figure  3.1-5)  is  designed  to  collect  data 
oi.  .  uipower  resource  expenditures  for  a  project  by  CPC  I,  WHS 


F i gu  re  3.1-5 


Resource  Expenditure  Data  Form 


element  or  other  aggregation  of  data.  For  this  initial  data 
collection  effort,  it  allows  the  data  provider  to  supply  infor¬ 
mation  to  the  lowest  level  of  detail  that  is  available.  Aggre¬ 
gation  of  the  data  to  appropriate  levels  for  inclusion  in  the 
data  base  was  performed  after  the  receipt  of  the  formats.  If 
this  format  is  used  for  collection  of  data  on  future  efforts, 
the  WBS  items  to  be  included  can  be  specifically  identified  as 
a  requ i rement . 

The*  format  is  structured  t  e  collect  data  by  month 
relative  to  the  contract  award  or  project  start  date.  For 
ongoing  projects,  the  last  month  for  which  actual  data  are  avail 
able  is  indicated;  estimated  data  are  used  for  the  balance  of 
the  development  project.  In  order  to  minimize  the  data  conver¬ 
sion  required  on  the  part  of  the  format  preparer,  manpower  data 
can  be  included  in  any  units  as  long  as  the  basis  for  the  units 
is  identified,  for  example,  manmonths  representing  1  r>2  manhours 

of  effort. 

la.  format  is  used  for  adding  projects  to  the  data 
base  and  up  to  five  wn.,  elements  with  60  months  of  data  can  In- 
entered  on  each  form.  It  does  •;  contain  any  entries  required 
for  cost  model  inputs;  however,  it  ~  likely  be  useful  for 
updating  cost  estimates  for  ongoing  projec. 

o.l.  5  CPC  I  Function  and  Sizing  Data  Detail 

This  format  (Figure  ii.  1-6)  is  designed  to  collect  soft¬ 
ware  size  and  functional  data  to  the  lowest  level  of  detail 
available.  It  constitutes  a  decomposition  of  a  CPC1  down  to  t ht 
module  level.  The  function  definition  is  selected  from  a  table 
of  electronics  systems  software  functions  contained  in  the  in¬ 
structions.  Since  this  table  is  not  all-inclusive,  the-  f  o  rma  t 
preparer  can  specify  a  unique  function  category  for  an  element  . 


Lines  of  source  code,  excluding  comments 
Words  of  CPU  memory  occupied. 


VIWWLW.I.* 


Size  information  is  requested  in  lines  of  source  code  because 
that  is  the  measure  currently  used  by  most  models.  Words  of 
memory  occupied  are  also  requested  so  that  object  instructions 
can  be  estimated,  since  most  sizing  analyses  performed  during 
system  development  are  done  in  memory  words.  Moreover,  PRICE-S 
requires  size  data  as  executable  machine  instructions,  which 
can  be  more  accurately  derived  from  the  memory  word  count  than 
from  lines  of  source  code.  The  language  in  which  each  subele¬ 
ment  is  programmed  is  also  identified  on  this  format. 

This  format  is  prepared  for  each  CPC1  and  documents 
each  subelement  into  which  the  CPC1  is  decomposed.  The  primary 
purpose  of  these  data  are  for  use  in  software  sizing  models. 
They  can  also  be  used  to  cross-check  the  size  distributions  on 
the  CPC1  Summary  Format  and  to  evaluate  the  memory  utilization 
efficiency  of  different  languages. 

1.1. b  Data  Collection  Format  Instructions 

The  data  collection  format  instructions  begin  with  a 
description  of  the  objective  of  the  data  collection  effort  and 
the  five  data  collection  formats.  A  separate  section  of  in¬ 
structions  for  each  format  follows  with  1 t  cm- by - i t  cm  instruc¬ 
tions.  Definitions  of  specific  data  items  are  part  of  the 
1  t em  instruction,  if  appropriate.  Subjective  items  include  a 
detailed  set  of  guidelines  drawn  from  cost  model  instructions 
to  (militate  consistent  evaluations  across  a  range  of  projects. 

A  glossary  del  ines  any  software  engineering  terms  used 
in  the  instructions,  as  well  as  software  development  milestones 
and  reviews.  This  glossary  was  developed  using  the  draft  M 1  I . - 
STb-SDS  (Kef.  2d)  as  the  primary  source  of  definitions;  this 
glossary  should  be  revised  to  reflect  the  final  version  of  the 
standard  when  it  is  issued.  Additional  items  were  taken  from 
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Rets.  25  and  30  through  33.  Finally,  Refs.  2,  3,  4  and  5  were 
used  for  model  specific  definitions. 

The  instruction  package  also  contains  a  table  of  stan¬ 
dard  software  function  categories,  developed  under  the  SARE 
effort,  to  be  used  in  completing  the  CPCI  Function  and  Sizing 
Detail  Format  and  the  recommended  work  breakdown  structure  to 
be  used  for  the  Resource  Expenditure  Form. 

Although  the  instruction  package  is  sizable,  it _ is  the 

minimum  require d  t  o _ i  n  s ure  consistent  interpretation  of  the  data 

requi rements ■  It  eliminates  much  of  the  additional  explanation 
that  is  given  verbally  when  detailed  instructions  are  not  avail¬ 
able  in  writing.  The  initial  feedback  from  industry  has  been 
that  the  package  is  quite  clear  and  wel 1 -conceived .  Residual 
questions  from  participants  in  the  data  collection  have  been 
re  1  a t i ve 1 y  few. 

3.2  COLLECTION  APPROACHES 

Three  different  approaches  were  selected  for  the  initial 
data  collection: 

•  Me t  hod  _1  --  Data  collection  forms  com¬ 
pleted  by  defense  contractor 

•  Method  2  --  Data  forms  completed  by  pro¬ 
gram  off  ice 

•  Method  3  --  Data  forms  completed  by  TASC 
analyst  using  program  office  documentation. 

1  he  relative  efficiency  of  these  approaches  was  evaluated  when 
t  he  data  collection  formats  were  completed. 
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Method  1  depends  on  the  willingness  of  companies  to 
divulge  proprietary  data  about  Government  software  projects  to 
a  third  party.  On  this  project  ,  cooperation  of  the  defense 
industry  was  obtained  through  the  use  of  previously  established 
professional  contacts.  Because  of  TASC ' s  corporate  policy  of 
not  contracting  with  defense  contractors,  the  participants  had 
no  concerns  about  a  possible  conflict  of  interest.  Typically, 
after  the  initial  contact  was  established  and  a  commitment  to 
furnish  data  secured,  constant  follow-up  activity  was  required 
to  ensure  success.  This  follow-up  activity  included  a  visit  to 
the  contractor  facility  to  gather  feedback  about  the  data  col¬ 
lection  formats,  to  identify  areas  of  ambiguity,  and  to  evaluate 
the  quality  of  the  data  furnished.  Each  participant  was  offered 
an  aggregated,  non  -  a t t r i bu t ab 1 e  tabulation  of  the  data  furnished 
by  ail  of  the  participants  for  use  in  evaluating  his  own  data. 

Method  2  is  dependent  on  establishing  contact  in  the 
program  office  with  an  appropriate  individual  familiar  with 
t  he  software  effort  for  the  project.  The  reward  offered  for 
participation,  that  is,  improved  software  cost  estimating  sup¬ 
port  in  the  future,  is  less  tangible  for  this  approach.  There¬ 
fore,  the  follow-up  activity  must  fie  even  greater  than  with 
the  first  approach.  Since  the  data  sources  at  the  program 
office  are  limited  primarily  to  the  existing  data  reporting 
i  t  c*ms.  »  the  program  office  contact  must  interact  extensively 
with  the  development  contractor,  especially  for  personnel  char¬ 
iot  e  r  t  s  t  i  c  and  manpower  utilization  data. 

Method  \  is  dependent  on  the  quality  and  completeness 
of  doe umen t a t i on  ivailable  for  a  project.  S i nee  the  analyst 
performing  the  data  extraction  is  not  usually  familiar  with  the 
program,  the  process  can  tie  very  time  consuming  and  not  all  of 
the  data  elements  required  are  available  in  the  documentation. 
Therefore,  add i t  i on a  1  effort  was  required  to  obtain  missing 
data  from  the  program  office  or  the  software  developer. 


All  three  approaches  were  used  on  this  effort  ,  although 
not  with  the  same  emphasis.  The  majority  of  the  data  was  ob¬ 
tained  through  the  defense  contractors,  because  they  have  more 
direct  access  to  the  data,  minimizing  the  potential  for  errors 
and  misinterpretation.  Table  3.2-1  summarizes  each  approach 
with  its  prerequisites,  advantages  and  disadvantages  based  on 
the  results  of  this  effort. 

3.3  DATA  BASE  MAINTENANCE  AND  GROWTH 

The  data  collected  under  this  effort  is  documented  in 
Volume  II  of  this  report.  Although  it  provides  a  solid  basis 
for  initial  analysis  and  model  calibration,  the  continued  evo¬ 
lution  of  the  software  development  process  and  the  need  for 
additional  data  points  to  permit  more  narrow  definition  of 
homogeneous  groups  requires  a  dynamic  data  base.  Opport un i t  i  es 
currently  exist  for  immediate  additions  to  the  data  base;  how¬ 
ever,  they  must  be  supplemented  with  a  mechanism  for  regular 
data  collection  during  the  system  acquisition  process. 

3  3.1  Dat  a  Co  1  led  i on  Met  hodu  1  ogy 

There  are  four  methods  that  can  be  employed  for  adding 
projects  to  the  data  base  created  under  this  initial  effort: 

•  Use  the  final  version  of  the  data  collec¬ 
tion  package  developed  under  this  effort 
"as  is"  (Appendix  B) 

•  Create  a  formal  Data  Item  Description 
(DID'  f  roii  the  collection  package 
developed  under  this  effort 

•  Formally  modify  existing  or  planned  data 
item  descriptions  to  include  the  data 
elements  specified  in  the  data  (rase-  design 
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•  Prepare  instructions  to  be  used  for  tailor¬ 
ing  existing  data  items  to  report  data 
specified  in  the  data  base  design. 

The  first  method  is  appropriate  for  collecting  data  t o  derive 
the  input  parameters  for  the  cost  models  used  t o  develop  an 
estimate  or  t o  support  a  source  selection  and  to  add  completed 
projects  to  the  data  base.  This  collection  package  serves  as 
a  replacement  for  ASD  Form  169a,  since  it  includes  not  only 
the  data  collected  on  that  form,  but  also  additional  data  for 
cost  models  which  are  not  covered  by  Form  169a.  The  last  three 
methods  are  applicable  for  collecting  data  on  new  projects  tor 
the  data  base  or  for  monitoring  performance  on  those  projects. 

Implementation  of  the  first  two  methods  is  based  on 
the  final  version  of  the  formats  resulting  from  the  feedback 
from  the  initial  collection  effort.  If  the  formats  are  not 
going  to  be  used  as  formal  contract  data  requirement  list  (CURL) 
items,  the  first  method  is  used.  Local  form  numbers  are  as¬ 
signed  and  the  package  is  ready  for  use.  On  the  other  hand .  if 
the  formats  arc-  going  to  be  used  as  contractual  deliverables, 
then  they  must  be  formally  established  as  data  items  within 
the  required  review  and  approval  cycle. 

The  last  two  methods  are  implemented  by  identifying 
the  specific  data  items  that  are  appropriate  lor  reporting 
subsets  of  the  data  elements  contained  in  the  data  collection 
formats.  In  Ref.  '35,  the  preliminary  report  for  this  effort, 
the  following  list  of  d  at  a  items  with  recommended  changes  was 
presen  t  ed : 

•  (lost  Iv  r  f  o  nuance  Reports  ( It  I  -  F  -  6000(1  and 
I)  1  -  1 '  -  60  1  (  '  Mod  i  i  \7  t  a  i  1  o  r  I  o  ran  a  l  1  of 
the  (lost  Is  ■  r  I  <  i  nil.  me  *■  Report  and  trie  (lost 
Schedule  Status  Report  to  show  maupowet 
data  for  '.re  soli  ware  WBS  elements  or 


format  9  of  the  Cost  Performance  Report 
to  show  manpower  by  WBS  rather  than 
functional  area 

•  Program  Master  Schedule  (DI-A-3007  and 
DI-A-3009)  -  Identify  specific  software 
development  milestones  at  the  system  and 
CPCI  levels  to  be  included  in  the  schedule 
submi ss ions 

•  System  Specifications  (DI-E-3101A,  DI-E- 
3102A  and  DI-E-3117)  -  Add  a  format  sum¬ 
marizing  mission,  functional,  hardware 
component,  and  software  component  data 

•  Software  Development  Specification  (DI-E- 

3119A)  -  Add  a  format  summarizing  the 

major  software  functions  and  the  CPCIs 

•  Software  Product  Specification  (DI-E- 

3120A)  -  Require  functional  categorization 

by  standard  category  as  part  of  the  module 
descriptions 

•  Software  Sizing  and  Timing  Analysis  Report 

(Dl-S-3581)  -  Require  sizing  information 
at  the  module  level  in  source  lines  of 
code,  as  well  as  CPU  memory  words;  incor¬ 
porate  a  format  identifying  the  hardware 
configuration  baseline  against  which  the 
analysis  is  made  and  include  information 
about  expansion  capability  and  reserve 
requi rements 

•  Software  Development  Plan  (R-D1D-103)  - 

Require  a  summary  format  identifying  tools 
and  techniques  to  be  used,  development 
computer  characteristics  and  constraints, 
and  an  experience  and  quality  profile  of 
personnel  assigned  to  the  project  . 

If  these  changes  are  incorporated,  most  of  the  data  elements 
required  for  the  data  base  will  be  available  directly.  The 
balance  can  be  derived  through  analysis  and  aggregation  of 
detail  data  from  the  reports. 


Formal  modification  of  the  existing  data  items  will 
require-  the  full  review  and  approval  cycle,  while  instructions 
tor  tailoring  the  data  items  can  be  implemented  locally.  How¬ 
ever,  since  the  Joint  Logistics  Commanders  (JLC)  Joint  Policy 
Coordinating  Group  on  Computer  Resource  Management  is  in  the 
process  of  developing  revised  data  item  descriptions  (Ref.  3A  ) 
for  reporting  on  software  projects,  formal  modification  ol 
existing  data  items  which  are  to  be  replaced  by  the  JLC  ver¬ 
sions  is  not  necessary.  Instead,  ESD  should  work  to  have  their 
requirements  incorporated  into  the  JLC  versions.  Separate 
action  will  only  be  required  to  modify  the  cost  performance 
data  items.  'fable  3.3-1  summarizes  the  recommended  changes  to 
the  JLC  data  item  descriptions  required  to  maintain  the  LSD 
Software  Data  Base. 

The  Program  Master  Schedule  data  item  does  not  need 
mod i f i ca t i on ;  however,  the  appropriate  milestones  and  levels 
of  detail  required  for  software  must  be  specified  in  the  con¬ 
tract.  The  changes  to  the  other  data  items  are  required  in 
order  to  insure  that  the  data  are  provided  consistently  from 
one  project  to  another  and  can  easily  be  extracted  for  incor¬ 
poration  into  the  data  base.  The  summary  formats,  which  are 
to  be  added,  should  be  standard  formats  with  detailed  instruc¬ 
tions  equivalent  to  those  prepared  under  this  effort.  These 
formats  should  be  augmented  with  standard  tables  of  keywords 
for  classifying  systems,  such  as  Table  B-2  in  Appendix  B  to 
this  report.  Table  B-2  itself  needs  to  be  further  refined  to 
include  other  types  of  systems. 

3.3 .2  Additional  Data  Sources 

Since  t h i s  effort  consisted  of  only  an  initial  collec¬ 
tion  to  develop  a  basic  c  a  pa  b i  1  i  t  y  and  field  test  the  co 1  1 ec  t  i on 
approach,  several  potential  data  sources  which  were  identified 
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SOFTWARE  DATA  ITEMS  (Continued) 
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during  the  study  were  not  actively  pursued.  These  sources  can 
be  used  to  substantially  increase  the  size  of  the  data  base  in 
the  near  term. 

Although  ESD  program  offices  were  used  as  data  sources 
on  a  limited  basis,  the  bulk  of  the  data  was  collected  from 
defense  contractors  in  order  to  concentrate  on  sources  to  which 
ESD  does  not  have  easy  access.  Therefore,  ESD  cost  analysis 
personnel  can  collect  data  directly  from  those  program  offices 
which  have  ongoing  or  recently  completed  programs  and  were  not 
used  as  sources  during  this  effort,  for  example,  SPADOC ,  WIS, 
and  Berl in  Radar . 

In  October  1983,  The  Aerospace  Corporation  completed  a 
software  sizing  survey  (Kef.  28)  for  the  Air  Force  Space  Divi¬ 
sion  (SD)  which  contains  size  and  function  data  for  ten  systems 
In  addition  to  the  sizing  data.  Aerospace  collected  many  of  the 
same  data  elements  that  are  included  on  the  Project  Summary, 
CPC1  Summary,  and  Resource  Expenditure  Forms  developed  under 
this  effort.  Although  many  of  the  systems  in  the  SD  data  base 
are  space-borne,  there  are  similarities  in  complexity,  relia¬ 
bility  requirements,  documentation,  and  timing  and  sizing  con¬ 
straints,  as  well  as  the  development  environment,  to  many  of 
the  embedded  systems  developed  for  ESD.  A  coordinated  effort 
with  SD  to  collect  the  missing  data  elements  and  to  establish 
a  data  sharing  agreement  for  the  future  would  result  in  sub¬ 
stantial  benefits  to  ESD. 

Finally,  there  are  many  other  organizations  within  the 
DoD  who  are  involved  in  software  development  and  are  in  the 
process  of  developing  software  data  bases.  Data  sharing  agree¬ 
ments  with  these  organizations  can  yield  additional  relevant 
data  points  for  the  ESD  data  base*. 


4.  ADVANCING  THE  STATE-OF-THE-ART  OF  SOFTWARE 

COST  ESTIMATION 

The  development  of  an  ESD  software  cost  data  base  is 
only  the  first  step  necessary  to  enhance  the  software  cost 
estimation  capability  at  ESD.  These  data  can  be  used  to  fine 
tune  existing  cost  estimating  techniques  and  Lo  develop  new 
tools  based  on  ESD  experience. 

4.1  DATA  BASE  AUTOMATION 

A  data  base  with  as  many  distinct  elements  per  project 
as  the  one  developed  under  this  el  fort  becomes  very  unwieldy 
even  when  it  contains  a  small  number  of  projects.  As  the  data 
base  grows,  the  update  and  maintenance  tasks  require  more  effort 
and  data  retrieval  becomes  a  tedious,  time-consuming  task.  Man¬ 
ual  transfer  of  data  into  model  calibration  or  model  devel¬ 
opment  format  can  introduce  errors  which  may  invalidate  the 
results.  Automation  of  the  data  base  can  eliminate  many  of 
these  problems  and  significantly  increase  the  useability  of 
the  data  base . 

The  automated  data  base  should  have  the  following 
capab i 1 i t i es : 

•  User  friendly  data  entry  with  error 
checki ng 


Data  base  editing  and  update 


Automatic  data  base  back-up 
Keyword  search  and  retrieval 


•  Input  file  generation  for  statistical 
packages  and  software  cost  estimating 
models . 


The  keyword  search  and  retrieval  system  coupled  with  the  ability 
to  generate  input  files  to  models  will  minimize  the  resources 
required  to  assemble  homogeneous  groups  of  data  for  model  vali¬ 
dation,  calibration,  and  development. 


4.2  COST  MODEL  VALIDATION  AND  CALIBRATION 

Because  cost  models,  such  as  PRICE-S  and  JS-1,  are 
based  on  specific  sets  of  data  that  may  not  be  representative 
of  ESD  software  developments,  they  should  be  validated  for  use 
by  ESD.  Using  the  data  base  developed  under  this  effort,  ESD 
should  evaluate  the  COCOMO ,  JS-1,  SLIM,  and  PRICE-S  as  predic¬ 
tors  of  cost  for  software  developments  representative  of  ESD 
experience . 

Initially,  the  ESD  data  base  should  be  compared  with 
the  data  used  to  develop  the  cost  models  which  are  to  be  val¬ 
idated.  The  major  similarities  and  differences  should  be 
identified  to  facilitate  analysis  of  model  outputs.  The  input 
parameters  for  these  models  are  extracted  from  the  historical 
data  so  that  the  predicted  cost  and  schedule  can  be  compared 
with  the  actual  cost  and  schedule  data.  The  results  are  then 
analyzed  to  identify  the  cause  of  any  anomalies  and  determine 
the  accuracy  of  the  models.  The  time-phased  resource  expen¬ 
diture  data  generated  by  the  models  should  also  be  compared 
with  the  historical  data  to  determine  its  validity. 


If  a  model  is  found  to  be  suitable  for  estimating  KSD 
software  development  cost  and  schedule,  it  is  calibrated  for 
homogeneous  subsets  of  the  ESD  data  base. 

4.3  ESD- UN  I QUE  MODEL  DEVELOPMENT 

The  validation  effort  described  in  the  previous  section 
may  result  in  the  identification  of  serious  deficiencies  in 
the  commercially  available  models.  If  the  calibration  feature 
of  the  models  cannot  compensate  for  the  differences  between  ESD 
developments  and  the  projects  represented  in  the  model  data 
bases,  the  development  of  a  model  unique  to  ESD  is  required. 

Development  of  the  model  begins  with  a  statistical 
analysis  of  the  data  base  to  identify  the  best  predictors  of 
electronics  system  software  development  costs.  Selection  of 
model  variables  would  concentrate  on  those  data  items  that  are 
available  to  the  estimator  or  can  reasonably  be  predicted  from 
historical  data.  Anyone  who  has  ever  developed  an  estimate 
is  sensitive  to  problems  of  data  availability  to  support  esti¬ 
mates. 

Based  on  the  results  of  the  data  base  analysis,  a 
comprehensive  software  cost  estimating  model  would  be  devel¬ 
oped  with  the  following  features: 

•  Cost  estimating  relationships  which  account 
for  variations  in  software  development  cost 
due  to  the  characteristics  and  requirements 
of  the  system,  to  the  expected  development 
team  profile,  and  to  the  development 

env i ronmen t 

•  Mi s t or i ca 1 1 y- based  resource  expenditure 
profiles 


•  Impact  assessment  of  resource  constraints 

•  Technology  adjustment  factor 

•  Sensitivity  analysis  mechanism 

•  Capability  to  develop  confidence  intervals 
for  the  estimate 

•  Cos t /'sc hedu  1  e  risk  assessment 

•  Data  base  interface  for  size  and  techni¬ 

cal  definition  by  analogy. 

With  this  tool,  a  cost  estimator  could  take  a  detailed  technical 
description  of  a  software  development  program,  determine  the 
software  size  and  technical  charact eri st ics  by  analogy,  specify 
a  development  environment  or  use  ESD  historical  environmental 
charac t eri s t ics  ,  and  predict  a  software  development  cost  and 
schedule.  He  could  then  perform  sensitivity  analysis  based  on 
different  assumptions  about  the  nature  of  the  software,  the 
development  environment  ,  and  resource  constraints.  Finally, 
since  the  model  is  specifically  tailored  to  ESD  software,  he 
would  have  greater  confidence  in  the  results  of  the  analysis. 

4 . 4  SOFTWARE  SIZING  METHODOLOGY 

Delivered  source  lines  of  code  continues  to  be  the 
most  often  used  metric  for  software  cost  estimating.  There¬ 
fore,  the  quality  of  an  estimate  can  be  significantly  improved 
with  better  sizing  tools. 

The  ESD  cost  data  base  can  be  used  as  a  verification 
tool  for  engineering  estimates  of  software  size: 

•  Using  size  ranges  for  categories  of  soft¬ 
ware  for  gross  level  confidence  checks 
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Using  a  catalog  of  standard  software 
modules  for  ESD  product  categories  to 
validate  both  the  sizing  and  technical 
completeness  of  the  engineering  estimate 


In  addition  to  confidence  checks,  the  data  base  can  be  used  to 
derive  a  size  estimate  by  matching  system  functional  require¬ 
ments  with  the  keyword  classifications  of  the  elements  in  the 
data  base.  This  analogy  technique  is  used  at  the  lowest  level 
of  detail,  system,  CPCI,  or  modu 1  e/un i t  ,  that  the  available 
system  definition  wi 1 1  allow. 


In  order  to  normalize  the  data  among  different  program¬ 
ming  languages  and  increase  the  effective  size  of  the  data  base, 
the  relationship  between  delivered  source  lines  of  code  and 
size  in  memory  words  occupied  can  be  used  to  develop  conversion 


ratios 


4.5  SOFTWARE  MAINTENANCE  COST  MODEL 


Although  this  data  collection  effort  was  primarily 
concerned  with  software  development  cost  estimating,  some  pre 
liminary  research  into  software  maintenance  cost  estimating 
was  conducted.  Since  software  maintenance  represents  a  sig¬ 
nificant  portion  of  the  total  life-cycle  cost  of  a  system,  a 
model  for  estimating  these  costs  should  be  developed. 


The  following  technical  objectives  must  be  achieved 
before  a  model  can  be  developed: 


Definition  of  the  basic  elements  of  the 
software  maintenance  process  including 
the  software  maintenance  facility,  the 
maintenance  workload,  the  testing  re¬ 
quirements,  the  configuration  control 
and  documentation  requirements 
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•  Identification  of  the  potential  variables 
for  use  in  modeling  these  basic  elements 
of  the  process 

•  Identification  of  ESD  computer  system 
charac ter i st ics  which  may  be  relevant  to 
the  maintenance  process 

•  Postulation  of  the  re  1  a t i on sh i ps  among 
the  element  variables  and  the  system 
characteristic s/requi remen t  s . 

The  above  activities  will  form  the  basis  for  establishing  the 
maintenance  cost  data  base  requirements  and  appropriate  work 
breakdown  structure.  Following  a  data  collection  effort  at  Air 
Force  software  maintenance  facilities,  the  resulting  data  base 
can  be  analyzed  s  t  a  t  i  s  t  i  ca  1  1  y  and  a  model  can  be  developed 
based  on  the  best  predictor  variables. 


r)  •  SUMMARY  AND  RECOMMEND  AT  I ONS 

A  comprehensive  data  collection  package  was  developed 
under  this  effort  and  refined  as  a  result  of  the  feedback  from 
the  initial  data  collection.  In  the  near-term,  this  data  col¬ 
lection  package  can  be  used  for  obtaining  information  for  com¬ 
pleted  or  ongoing  projects  and  for  cost  model  input  parameters. 

For  the  long-term  a  formalized  method  for  data  collec¬ 
tion  is  required  to  maintain  the  data  base  and  ensure  its  con¬ 
tinued  usefulness.  It  is  recommended  that  ESD  become  involved 
in  the  current  Joint  Logistics  Commanders  effort  to  revise  many 
of  the  existing  software  data  items  and  incorporate  summary 
data  formats  into  those  data  items.  While  these  revised  data 
items  are  in  the  review  rind  approval  cycle,  the  tailored  data 
item  approach  should  be  used  on  software  development  efforts 
for  which  contracts  will  be  awarded  in  the  near-term. 

In  addition  to  the  data  that  can  be  obtained  for  LSI) 
programs,  the  size  of  the  data  base  can  be  significantly 
increased  through  data  sharing  arrangements  with  other  bob 
agencies.  I'he  Air  Force  Space  bivision  would  be  a  good  start¬ 
ing  point. 

1  he  availability  of  data  is  the  crucial  tirst  step 
for  adva  ru  r  rig  the  state-of-the-art  of  software  cost  estimation. 
1 igure  a- 1  illustrates  a  road  map  for  increasing  the  size  of 
tie  data  ( ) a •  and  using  it  to  improve  existing  techniques,  ns 
welt  as  for  developing  new  tools.  Future  efforts  should  con- 
i  •  lit  rate  on  : 
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ACQUISITION  PHASE 


MAINTENANCE  PHASE 


DATA  BASE  GROWTH 
ACTIVITIES 


DATA  BASE 
AUTOMATION 


SOFTWARE  MAINTENANCE 
DATA  BASE  DEVELOPMENT 


COST  MODEL  VALIDATION  AND  CALIBRATION 
SIZING  METHODOLOGY  DEVELOPMENT 


SOFTWARE  MAINTENANCE  MODEL 
DEVELOPMENT 


ESD-UNIQUE  MODEL  DEVELOPMENT 


Figure  5-1  Software  Life-Cycle  Cost  Estimation  Road  Map 
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The  potential  offered  by  the  data  availability  makes  ESI)  a 
leader  in  the  st ate-of - t he-art  of  software  cost  estimating. 


APPENDIX  A 


SOFTWARE  COST  MODEL/DATA  COLLECTION  FORMAT 
CROSS  REFERENCE  TABLES 


The  following  tables  cross  reference  the  cost  mode  1  in¬ 
put  parameters  to  specific  items  on  the  data  collection  formats 
contained  in  Appendix  B.  Many  of  the  items  can  be  input  directly 
into  the  cost  models,  while  others  are  derived  from  several  data 
items  using  estimator  judgement.  Validation  and  calibration 
activities  would  require  the  same  input  parameters  for  several 
homogeneous  CPCIs. 


TABLE  A- 1 

COCOMO  MODEL  DATA  REQUIREMENTS 


INPUT  PARAMETER 
Development  Mode  Selection 

Software  Size 

Required  Software  Reliability 
Data  Base  Size 
Product  Complexity 
Execution  Time  Constraint 

Main  Storage  Constraint 

Virtual  Machine  Volatility 

Computer  Turnaround  Time 

Analyst  Capability 
Applications  Experience 
Programmer  Capabi 1 i ty 
Virtual  Machine  Experience 
Programming  Language  Experience 
Modern  Programming  Practices 

Use  of  So  f  t  wa re  Too  1 s 


DATA  COLLECTION  FORMAT  REFERENCE 


Project  Summary  Form  Items  3.1, 
3.2,  3.7,7 

CPCI  Summary  Form  Items  6.1A,  8 
Development  and  Target  Computer 
Form  I  terns  1.7,  1.8 

CPCI  Summary  Form  Items  8.1,  8.9 
Function  &  Sizing  Detail  Form 

CPCI  Summary  Form  Item  5 

CPCI  Summary  Form  Item  8.3 

CPCI  Summary  Form  Item  6 

Development  and  Target  Computer 
Data  Form  Items  1.4,  1.6,  1.9B 

Development  and  Target  Computer 
Data  Form  Items  1.2,  1.3,  1.5, 
1.9,  1.10 

Development  and  Target  Computer 
Dat  a  Form  Item  1.8 

Development  and  Target  Computer 
Data  Form  Item  2.2 

CPCI  Summary  Form  Item  4.2 

CPCI  Summary  Form  Item  4.1A 

CPCI  Summary  Form  Item  4.2 

CPCI  Summary  Form  Item  4. ID 

CPCI  Summary  Form  Item  4. 1C 

Project  Summary  Form  Item  4 
CPCI  Summary  Form  I  t  ('in  4.  IB 

Project  Summary  Form  Item  5 
CPCI  Summary  Form  Item  4. IE 
Development  and  Target  Computer 
Data  Form  Item  2.8 


TABLK  A- 1 

COCOMO  MODEL  DATA  REQUIREMENTS  (Continued) 


INPUT  PARAMETER 

DATA  COLLECTION  FORMAT  REFERENCE 

Required  Development  Schedule 

Project  Summary  Form  Item  b 

CPCI  Summary  Form  Item  3 

Development  Manmonths 

For  va 1 i dat  i on/ca 1 i bra  t  i on 
Resource  Expenditure  Form 

Schedule  Duration 

For  va 1 i da t i on/ca 1  i bra t  ion 

Project  Suirimary  Form  Item  6 

CPCI  Summary  Form  Item  3 

TABLE  A- 2 

JS-1  MODEL  DATA  REQUIREMENTS 


INPUT  PARAMETERS 
Software  Size 

Interactive  Environment  Rating 
Resource  Rating 
Tool  Quality  Rating 

Project  Complexity  Values 


Response  Requ i remen t s 

Source  Statement  Type  Mix 

Development  Factor 

Special  Display  Requirements 

Detailed  Definition  of 

Operational  Requ i remen t s 

Real  Time  Ope  rat  ion 

CPU  'lime  Constraint 


DATA  COLLECTION  FORMAT  REFERENCE 


J  CPCI  Summary  Form  Items  8.1,  8.9 
i  Function  &  Sizing  Detail  Form 

i  Development  and  Target  Computer 
Data  Form  Item  2.9 

i  Development  and  Target  Computer 
Data  Form  Item  2.2 

Project  Summary  Form  Item  5 
|  CPCI  Summary  Form  Item  4. IE 
Development  and  Target  Computer 
1  Data  Form  Item  2.8 

I 

Project  Summary  Form  Items  3.1, 

|  3.2 ,  3.3,  3.4 
CPCI  Summary  Form  Items  8.4, 

8.5,  8.6,  8.9 

!  Development  and  Target  Computer 
!  Data  Form  Items  1.7,  1.8 

i 

i  CPCI  Summary  Form  Item  8.  > 

CPCI  Summary  Form  Item  8.6 

CPCI  Summary  Form  Item  8.9 

CPCI  Summary  Form  Item  8.7 

:  Project  Summary  Form  Item  8 
Estimator  Judgement 

CPCI  Summary  Form  Item  8 . 5 A 

Development  and  Target  Computer 
Dat a  Form  1 t  em  1.11 


CPU  Memory  Constraint  Development  and  Target  Computer 

Data  Form  I t  em  1.10 

First  Software  Developed  CPCI  Summary  Form  Item  4.11) 

on  CPU  Syst  em 


l.'om  urri'nl  ADP  Hardware 
Deve 1 opmen t 
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Development  and  Target  Computer 
Dat  .r  Form  I  t  ems  1.7,  1.8 


TABLE  A- 2 

JS-1  MODEL  DATA  REQUIREMENTS  (Continued) 


INPUT  PARAMETERS 

DATA  COLLECTION  FORMAT  REFERENCE 

Developer  Using  Remote  Computer 

Development  and  Target  Computer 

Data  Form  Items  2.6,  2.7 

Development  at  Operational  Site 

Development  and  Target  Computer 

Data  Form  Items  2.3,  2.9 

Development  Computer  Different 

Development  and  Target  Computer 

From  Target  Computer 

Data  Form  It ems  1.1,  2.1 

Development  at  Multiple  Sites 

Development  and  Target  Computer 

Data  Form  Items  2.5,  2.6,  2.7 

First  Use  of  Programming 

CPC I  Summary  Form  Items  U.  1C, 

Language 

8.8 

System  Type 

CPCI  Summary  Form  Item  5 

Documentation  Type 

CPC1  Summary  Form  Item  7 

TABLE  A- 3 

SLIM  MODEL  DATA  REQUIREMENTS 


INPUT  PARAMETERS 

DATA  COLLECTION  FORMAT  REFERENCE 

Software  Size 

CPCI  Summary  Form  Items  8.1,  8.9 
Function  &  Sizing  Detail  From 

Percent  of  Development 

On -line/ Interactive 

Development  and  Target  Computer 
Data  Form  Items  2.2,  2.3E 

Proportion  of  Development 

Computer  Dedicated  to  Effort 

Development  and  Target  Computer 
Data  Form  Item  2.9 

Proportion  of  Development 

Computer  Used  for  Other  Work 

Development  and  Target  Computer 
Data  Form  Item  2.9 

Proportion  of  System  Coded 
in  Higher  Order  Language 

CPCI  Summary  Form  Item  8.8 

Primary  Language  Used 

CPCI  Summary  Form  Item  8.8 

Software  System  Type 

Project  Summary  Form  Items  3.1, 
3.3',  3.9 

CPCI  Summary  Form  Items  2, 

8.9,  8.5 

System  Level 

Project  Summary  Form  Items  3.1, 
3.2',  3.3,  3.9 

CPCI  Summary  Form  Items  2,  8.9 

Target  Machine  Memory 

Utilization 

Development  and  Target  Computer 
Dat  a  Form  I t  ems  1.2,  1.3,  1.5, 

1 ,9A,  1.10 

Proport  ion  of  Real-Time  Code 

CPCI  Summary  Form  Item  8.5 

Modern  Programming  Practice 

Usage 

Project  Summary  Form  Item  9 

CPCI  Summary  Form  Item  9. IB 

Personnel  Experience 

CPCI  Summary  Form  Items  9.1A, 

9 . 1 C ,  9 . 1 D ,  9 . 1 E ,  9.2 

State  o 1  Technology  Factor 

_ 

Derived  By  Calibration  Using 

CPCI  Summary  Form  Items  3,  8.1 
Resource  Expenditure  Form 

-"AV1! 
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TABLE  A- A 

PRICE-S  MODEL  DATA  REQUIREMENTS 


INPUT  PARAMETERS 

— 

DATA  COLLECTION  FORMAT  REFERENCE 

Project  Magnitude 

CPCI  Summary  Form  Items  8.1, 

8.8,  8.9 

Function  &  Sizing  Detail  Form 
Development  and  Target  Computer 
Data  Form  Item  1.2 

Project  Application 

CPCI  Summary  Form  Item  8. A 

Level  of  New  Design  and  Code 

CPCI  Summary  From  Item  8.9 

Resource 

Derived  by  calibration  using 
Project  Summary  Form  Item  8 

CPCI  Summary  Form  Items  3,  A.l, 

A  .  2  .  8.1,  8 . A ,  8.8,  8.9 
Development  and  Target  Computer 
Data  Form  Items  1,  2.5 

Utilization 

Development  and  Target  Computer 
Data  Form  Items  1.2,  1.3,  l.A, 
1.5,  1.6,  1.9,  1.10,  1.11 

P 1  at  f orm 

Project  Summary  Form  Items  3.1  , 
3.2,  3.3,  3 . A ,  3.7 

Complexity  or  Schedule 

Project  Summary  Form  Item  8 

CPCI  Summary  Form  Items  3, 

A.l,  A . 2 

Development  and  Target  Computer 

Da  t  a  Form  I t  ems  1.7,  1.8,  2.5, 
2.6,  2.7 

New  Design 

CPCI  Summary  Form  Item  8.9 

New  Code 

CPCI  Summary  Form  Item  8.9 

Maximum  Man  loading 

CPCI  Summary  Form  Items  A. 8,  A. A 

Mix 

CPCI  Summary  Form  Item  8. A 

Interface  Types 

Project  Summary  Form  Item  3.2 

Interface  Qu  a  n  t  i t i e s 

Project  Summary  Form  Item  3.2 

_ 

APPENDIX  B 

DATA  CO ELECTION  PACKAGE 

This  appendix  contains  the  ESI)  software  cost  data 
base  data  collection  package,  which  consists  of  a  comprehen¬ 
sive  set  of  instructions  containing  reference  tables  and  a 
glossary.  Five  data  collection  formats  are  also  included. 


SOFTWARE  DATA  COLLECTION  FORM  INSTRUCTIONS 


INTRODUCTION 


The  objective  of  this  data  collection  effort  is  to 
develop  a  data  base  of  software  cost,  schedule,  sizing,  and 
technical  information  for  use  in  cost  model  validation  and 
ca 1 i b ra t i on ,  for  software  sizing,  and  for  cost  model  develop¬ 
ment  . 


There  are  five  different  forms  used  to  collect  the 
required  data: 


Software  Development  Project  Summary  Data 
Form  -  collects  data  at  the  project  level  to 
define  the  project  functional  and  technical 
characteristics,  the  development  tools  and 
methods  available,  the  project  schedule,  the 
documentation  required,  and  the  change  history. 
It  is  prepared  for  each  project  for  which 
data  is  provided . 

Development  and  Target  Computer  Data  Form  - 
collects  data  for  each  target  computer  (opera¬ 
tional  host)  in  the  system  and  for  the  devel¬ 
opment  computer  on  which  the  corresponding 
operational  software  is  developed. 

Computer  Program  Con f i gura t i on  Item  Summary 
Data  -  collects  information  at  the  CPC I  level 
to  define  the  development  schedule,  the  per¬ 
sonnel  characteristics  and  constraints,  the 
CPC1  size  and  characteristics,  documentation 
requirements,  and  the  failure/error  history 
during  the  development  .  This  form  is  prepared 
for  every  CPCI  identified  on  the  software 
development  project  summary  form. 

Resource  {Expenditure  Data  Form  -  collects 
timephased  manpower  utilization  data  at  the 
lowest  level  avai lable  for  the  project . 


Computer  Program  Configuration  Item  Function 
and  Sizing  Data  Detail  Form  -  collects  software 
size,  function  and  programming  language  infor¬ 
mation  at  the  lowest  level  available  for  each 
CPCI  listed  on  the  software  development  project 
summary  data  form. 


Detailed  instructions  for  completing  the  forms  are 
included  in  the  following  sections.  Attachment  A  is  a  glossary 
of  terms  used  in  the  forms.  Attachment  B  is  a  recommended 
work  breakdown  structure  for  software  development  projects. 
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SOFTWARE  DEVELOPMENT  PROJECT  SUMMARY  DATA 


1 .  Proj  ect  Name : 

2.  Devel  opmerit  Coat  ract or/Organ i  zat  ion  : 

3.  Project  Description 

3.1  Mission  Description:  _ 


3.2  Major  Hardware  Interfaces: 


3.3  Major  Svstem  Functions: 


3.4  Major  Software  Functions: 


3.5  Number  of  CPCIs: 

3.6  CPC  I  Names: 


3  .  7  System  User : 


Development  Contractor  Other  Commercial  Company  _ 

Department  of  Defense  Cither  Government  Agency 

4.  Development  Methodologies  Used 
4 . 1  Spec i f i cat i on : 


Fihm:  t  i  ona  1 


Procedural  English 


Other : 


4 . 2  Des ] gn : 


Top  Down  bottom  Up  Iterative  Enhancement 


Hardest  First  None 

4 .  i  Deve 1  opmerit  : 

Top  Down  Bottom  Up 

Hardest  first  None 

4 . 4  Cod i ng : 

S l mu  I  at  t  ng  Cons  t  ruct 
Other : 


Other: 


Iterative  Enhancement 


Other : 


Structured  Code 


wx:  •  ^  v  v: 


4.5  Testing: 

Top  Down(stubs)  Bottom  Up(drivers) 

Specification  Driven  Structure  Driven 

None  _  Other:  _ 

4.6  Va 1 idation/Ver if icat ion ( Inspect  ion) : 

Peer  Review  _ _  Walk  Throughs  _  Proof 

None  _  Other: 

4.7  Formalisms: 

Program  Design  Language (Speci fy) : _ _ 

HIPO  Charts  _  Flowcharts  _  Chapin  Charts  _  HOS 

Other : 

5.  Software  Development  Tools  Used: 

Assembler  Basic  Linker 


Basic  Monitor 
Higher  Order  Language 
Compiler  _ 

Basic  Source  Editor  _ 

Basic  Library  Aids 
Real-time  or  Timesharing 
Operating  System 
Interactive  Debug  Aids 
Interactive  Source  Editor 
Database  Design  Aid 
Performance  Measurement 
And  Analysis  Aids 
Set-Use  Static  Analyzer 
Basic  Text  Editor  &  Manager 
File  Manager 
Documentation  System 
Requirements  Specification 
Language  and  Analyzer 
Fault  Report  System 
Instruction  Set  Simulators 
Data  Entry  Control  Tools 
Conversion  Aids 
Other : 


Batch  Debug  Aids 
Macro  Assembler 

Simple  Overlay  Linker  _ 

Language  Independent  Monitor 

Basic  Data  Base  Aids  _ 

Extended  Overlay  Linker  _ 

Database  Management  System 

Simple  Programming  Support  Library  _ 

Virtual  Memory  Operating  System 

Simple  Program  Design  Language  _ 

Programming  Support  Library  With 
Basic  Configuration  Management  Aids 
Control  Flow  Static  Analyzer 
Program  Flow  and  Test  Case  Analyzer  __ 

Full  Programming  Support  Library  _ 

Project  Control  System 
Extended  Design  Tools 

Automated  Verification  System 
Cros  scomp i 1 e  r s 
Display  Formatters 
Commun ications  Processing  Tools 
Structured  Language  Tool 
Othe  r  : 


Other : 


Other : 


Development  Schedule* 


Original  Actual 

Project  Milestone  _ _ Date _ _ Da t e 

Contract  Award  _  _ 

System  Requirements  Review  _ _ _  _ 

System  Design  Review  _ _ _ 

Preliminary  Design  Review  _  _ 

Critical  Design  Revie*w  _ _ _ 

Preliminary  Qualification  Test 

Formal  Qualification  Test  _ _  _ 

Start  CPCI  Integration  Into  System  _  _  _____ 

Complete  CPCI  Integration  into  System  _  _  _ 

Start  Development  Test  &  Evaluation  _  _ _ 

Complete  Development  Test  R  Evaluation  _  _ 

Start  Initial  Operational  Test  R  Eva  1  _ _ 

Complete  Initial  Operational  Test  R  Eva  1  _ 

Functional  Configuration  Audit  _  __  _  _ 

Physical  Configuration  Audit 
Formal  Qualification  Review 

System  Delivery  _ 

Documenta l i on 


Document  Title  _ 

System  Engineering  Management  Plan 
Computer  Program  Development  Plan 
System  Test  Plan 
Other : 

Other: 

Other : 


//  of  Pages _  Est  '  d  or 


Est i mated 
Da  t  e 


Act  ’  1 


Jk 
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SOFTWARE  DEVELOPMENT  PROJECT  .SUMMARY  DATA  FORM  INSTRUCTIONS 

Item  1  :  Enter  the  name  of  the  project  and  the  date 

on  which  this  form  is  being  prepared. 

Item  2 :  Identify  the  company  or  organization  which 

is  actually  performing  the  software  design  and  development. 

1  tern  3.1:  Briefly  describe  the  overall  mission  or 

purpose  of  the  system  for  which  the  software  is  being  developed. 

I t  em  3 . 2  :  Identify  the  major  hardware  components 

with  which  the  software  will  interface,  for  example,  radars , 
communications  equipment  ,  sensors,  other  embedded  computer 
sys  t  ems ,  etc. 

1 t  em  3.3:  List  the  major  functions  performed  by  the 

system . 

Item  3.6:  List  the  major  lunct ions  performed  by  t he 

so  f  t  wa  re . 

Item  3.5:  Identify  the  number  of  Computer  Program 

Configuration  Items  (CPCIs)  into  which  the  system  is  divided. 

Item  3.6:  List  the  names  of  each  CPC  I  in  the  system. 

Item  3.7:  Indicate  with  an  X  the  user  for  whom  the 

system  is  being  developed. 

I  t  ems  6.1  -/1.7:  I  rid  i  cate  with  an  X  all  of  the  soft  wa  re 

development  methodologies  and  strategies  used  for  each  act  ivitv 
on  this  pro  j  ee  t  . 


3- 


I  t  eni  5 :  Indicate  with  an  X  all  of  the  software  devel¬ 

opment  tools  used  on  this  project;  use  the  last  four  items  to 
specify  other  tools  used  which  are  not  included  in  the  listing. 

I  tern  b :  Enter  the  original  schedule  date  for  each 

applicable  milestone  (enter  N/A  if  a  milestone  is  not  applic¬ 
able).  Although  these  milestones  represent  formal  contractual 
activities  in  the  Department  ot  Defense  software  acquisition 
process,  many  non-defense  projects  will  have  milestones  which 
are  equivalent  to  these,  e.g.,  contract  award  is  equivalent  to 
project  start  and  critical  design  review  is  equivalent  to  com¬ 
pletion  of  detail  design.  If  the  formal  milestones  are  not 
required  in  the  project  schedule,  data  for  equivalent  activities 
should  be  used.  Definitions  of  these  milestones  are  provided 
in  Attachment  A  of  these-  instructions.  Unless  otherwise  indi¬ 
cated,  the  date  should  reflect  the  activity  completion  date. 
Where  available,  enter  the  dclu.tl  date  of  completion  for  t  he 
milestone;  for  ongoing  efforts,  enter  the  current  estimate  for 
completion  of  the  milestone. 

I  t  phi  7:  Enter  the  number  of  pages  for  each  document 

listed  and  specify  any  additional  documentation  required  for 
the  software  at  the  project  level.  Indicate  with  an  X  in  the 
appropriate  column  whether  the  page  count  is  estimated  or 
actual  . 

Item  8:  Enter  the  number  of  requirements  changes 

which  occurred  during  each  completed  development  phase,  t  he 
in-t  increase/decrease  in  the  total  system  source  instruction 
count  and  t he  net  i no rease/dec lease  in  t he  estimated  manpower 
tor  the  software  (level  opine  n  t  effort. 


DEVELOPMENT  AND  TARGF'l  COMPUTER  DATA  FORM 


urget  Computer 

.1  Manufacturer:  Mode  1  Number: 

.  2  Main  Memo  rv  Size  in  Words:  Word  Size: 

.  i  Maximum  Main  Meinorv  Size 
•4  CP' ’  Processing;  Speed  ; 

.  >  Reserve  Memory  Requirement:  % 

.  b  Reserve  Timing  Requirement:  % 

1.7  Concurrent  Development  with  Software:  Yes  No 

I.K  Virtual  Mac  (line  Volatility: 

Verv  Low  I ow  Nominal  High  Very  High 

1  . Percent  Utilization:  'SIT  SI  -  70%  7 1  -8S%  Kb-MST 

A.  CPI  Memory 
H  .  Execution  1 l me 

1.10  CPI  Memory  Constraint  Evaluation: 

No  Memory  Economy  Measures  Required 
Some  Overlay  I se  ui  Segmentation  Required 
Extensive  overlay  And/Ot  Segmentation  Required 
Complex  Meinorv  Management  Economy  Measures  Required 

1.11  CPI  I i me  Constraint  Evaluation: 

No  Software  is  CPU  lime  Constrained 


JST  of 

Soiirc  e 

Code 

1  s 

T  i  me 

Cons t  r  a i ne 

sot  of 

Sou  roe 

Code 

1  s 

1  1  llle 

Cons t  r  a i nt 

7 ST  of 

Sou IC e 

Code 

1  s 

1  1  me 

Cons  t  r.n  in 

1  11'  Cl’CIs  Hosted  on  fill:.  C 'oliuul  t  e  r  ; 


Development  Computer 

2.1  Same  as  Target  Computer:  Yes  _  No  _ 

If  No,  Manufacturer: _ Model  Number:  _ _ 

Difference  Between  Development  and  Target  Computer:  _ 

2.2  Turnaround  Time: 

Low  (interactive,  specify):  _  Nominal  (<4hrs)  _ 

High  (4-12hrs)  _  Very  High  (>12hrs)  _ 

2.3  Percentage  of  Source  Instructions  Developed  Using  Each  of  the 
Following  Access  Modes  (Tota 1 = 1 00%) : 


A. 

Batch 

_ % 

B. 

Dedicated  Processor 

% 

C. 

Test  Bed  with 

High  Priority 

_ % 

D. 

Test  Bed  with 

Low  Priority 

_ % 

L. 

Interact i ve 

% 

F. 

Other: 

% 

2.4  For  Interactive  Development,  Indicate  the  Average  Number  of 
Software  Engineers/Programmers  per  Terminal: 

2.5  Number  of  Development  Sites 

2.6  Development  Site  Locations: 

2.7  Development  Computer  Lot  at i on ( s  )  : 


2.K  Software  Development  Tool  Usage 

Very  Low  Low  Nomin.il  High  Very  High 

2  .  lJ  Deve  1  opine n t  Computer  Kcsouu  c  Av.i  i  I  ,i h  i  I  i  t  v 


DEVELOPMENT  AND  TARGET  COMPUTER  DATA  FORM  INSTRUCTIONS 

I_t  em  1.1:  Identify  the  manufacturer  of  the  operational 

system  for  which  this  software  is  be i ng  developed.  If  the 
computer  is  an  off-the-shelf  item,  enter  the  model  number. 

1 1  em  1.2:  Enter  the  main  memory  size  in  words  and 
indicate  the  word  size  in  bits.  For  ongoing  projects,  this 
should  reflect  the  current  configuration  of  the  computer.  For 
completed  projects,  t h  i  s  entry  should  reflect  the  delivered 
configuration  of  the  computer. 

2_tem_l_.3:  Enter  the  maximum  main  memory  size  in  words 

that  can  be  attained  without  major  modification  to  the  current 
or  delivered  computer  configuration. 

liUlL-LJ* :  Indicate  the  CPU  processing  speed  in  in¬ 

structions  per  second. 

Item  1.5:  Enter  the  percent  of  Item  1.2  which  must 
be  left  available  for  expansion/growth  after  system  delivery. 

Item  _!.(>:  Enter  the  percent  of  the  total  processing 
time  which  must  be  left  available  for  expansion/growth  after 
sys t  em  de 1 i ve ry . 

I  U'lii  1.7:  Indicate  w  i  t  h  an  X  whether  the  target  vir¬ 

tual  machine  is  being  deVelop»-d  < -oncurrent  1  v  with  the  software. 

Item  1  .  d  :  Using  the  following  criteria,  indicate 

with  an  X  the  degree  of  volatility  in  the  design  of  the  virtual 
mac  h l ne : 


Very  Low  =  No  major  changes;  one  minor  change  every 
12  months 

Low  =  Major  changes  every  12  months;  minor  changes 
every  month 

Nominal  =  Major  changes  every  6  months;  minor  changes 
every  2  weeks 

High  r  Major  changes  every  2  months;  minor  changes 
every  week 

Very  High  -  Major  changes  every  2  weeks;  minor  changes 
every  2  days 

1 1  em  1 . 9A :  Indicate  with  an  X  the  maximum  percentage 
of  main  storage  used  by  any  group  of  CPC  Is  operating  concur¬ 
rent  ly . 

Item  1 . 9B :  Indicate  with  an  X  the  maximum  percentage 
of  processing  time  used  by  any  group  of  CPCIs  executing  con¬ 
current  ly . 

1 1  em  1.10:  Indicate  with  an  X  the  level  of  memory 
conservation  measures  required  to  satisfy  the  reserve  memory 
requirement  in  Item  1.5. 

I t  em  1.11:  Indicate  with  an  X  the  percentage  ol  the 
software  that  requires  special  coding  effort  to  enhance  timing 
performance . 

1 1  em  1.12:  Enter  the  names  of  the  CPCIs  which  are 
hosted  in  this  computer. 

Item  2. 1 :  Indicate  with  an  X  whether  the  development 

computer  is  the  same  as  the  target  operational  computer.  If 
the  computers  are  different,  identify  the  manufacturer  and 


model  number  of  the  development  computer.  Describe  any  differ¬ 
ences  between  the  two  computers  which  would  affect  the  software 
development  effort,  e.g.,  different  operating  systems,  computers 
compilers,  main  memory  and  timing  constraints.  Is  development 
of  a  target  computer  emulator  required? 

1  tern  2 .2 :  Indicate  with  an  X  the  average  turnaround 

time  for  the  development  computer.  If  a  rating  of  low  applies, 
specify  the  approximate  response  time  experienced  on  the  system 
per  computing  job,  e.g.,  a  unit  test  or  compile. 

I  t_em  2 :  Indicate  the  percentage  of  source  instruc¬ 

tions  developed  using  the  specified  access  modes.  Specify  any 
other  mode  used  and  its  percentage  of  utilization. 

It_em  2^4:  For  terminals  which  are  readily  accessible 
to  members  of  the  development  team,  indicate  the  average  number 
of  software  engineers  and  programmers  per  terminal. 

I  t  em  2 .1 :  Hitter  the  number  of  individual  sites  where 
this  software  is  be i ng  developed.  Indicate  any  site  that  is 
working  as  a  subcontractor  to  the  development  contractor. 

Item  2.b :  Identify  the  geographic  location  (city  and 

state)  o!  each  of  the  development  sites. 

Item  2.7:  Specify  the  location  of  the  development 
computers  used  to  deveiop  this  software  by  each  of  the  dev<  1- 
opmen t  sit  rs . 

Item  2.8:  Specify  the  degree  to  which  the  development 
tools  available  within  the  development  contractor's  organization 
are  used  in  the  development  of  t h i s  software. 
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Item  2.9:  Indicate  the  percentage  of  the  total  devel¬ 
opment  computer  capacity  that  is  available  for  work  on  this 
project.  This  percentage  should  reflect  the  impact  of  opera¬ 
tional  uses  or  other  developments  competing  for  the  use  of 
this  resource. 


COMPUTER  PROGRAM  CONFIGURATION  ITEM  SUMMARY  DATA 


1.  CPC I  Name:  _ 

2.  Functional  Description: 


3.  Schedule 

3.1  Mil estone  Data 

Original  Actual  Estimated 

CPC1  Development  Milestones  Date  Date  Date 

Design  Start 

Preliminary  Design  Review 
Development  Specification  Approval 
Critical  Design  Review 
Start  Coding/Debugging 
Comp  1 et e  Coding/Debugging 
Start  Informal  Integration  Test 
End  Informal  Integration  g  Test 
Preliminary  Qua J j f i  c  at i on  Test 
formal  Qua  1 i f i ca t i on  Test 
Product  Specification  Approval 
Functional  Configuration  Audit 
Physical  Configuration  Audit 

3.2  Schedule  Accel erut lon/St  ret c houl  Assessment: 


Below  7 

Personne 1 

A.)  Average  Experience 


A  .  App I  i  c  a  t  ion  Area 

B  .  Te (  hli  1  ipies.  Used 

C  .  Languages  Used 

I).  Vi  rt  ua  I  Mac  h  i  lie 

F.  .  Support  So  f  twa  r  c/Ton  I  s 


Ri>-  1  Ul'K. 


1  3  1  -  ltd)"'. 


I  mo  1  -Amos  A-12mos  l-3yrs  3~6vrs  tiers 
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4.2  Average  Personnel  Quality  Percentile: 

0-15%  16-35% 36-55% 

56-75%  _____  76-90% 90-100% 

4.3  Manpower  Availability:  _ % 

4.4  Peak  Manloading: 


5.  Reliability  Requirement: 

Very  Low 

6.  Complexity: 

Low 

Nominal  High 

Very  High 

Ve  ry 

Low 

Low 

Nomina  1 

High 

7 .  Qual i ty  of  Speci 

f i cat i on 

Very  High 

Extra  High 

Ve  ry 

Precise 

Precise 

1 mp  rec i se 

8 .  Size 

8.1  Deliverable  Lint's  of  Source  Code  Excluding  Documentation: 

8.2  Lines  of  Source  Code  Documentation: 

8.3  (lata  Base  Size  in  Bytes  or  Characters: 

8.4  Size  Breakdown  by  Operation  As  a  Percent  of  Item  8.1 (Total  =  1 


A. 

Data  Storage  8  Retrieval 

,  _ % 

B. 

Online  Communications 

_ % 

C . 

Real-Time  Command  8  Control 

% 

1). 

Interactive  Ope r a t  ions 

% 

E. 

Mathematical  Operations 

_  % 

E. 

String  Manipulation 

O/ 

G. 

Operating  Systems 

% 

H  . 

Other : 

% 

1 

Ot  her : 

o, 

H- 

8.5  Operation.il  Response  Requi  rements  Distribution  As  a  Percent  of 

Item  8 .  1 ( To t  a  1  =  1 00% ) : 

A.  Beal -Time  %  B.  On-Line  % 

C.  Time-Constrained  _  %  D.  Non-Time  Critical  % 

8.6  Source  Statement  Type  Mix  As  a  Percent  of  Item  8.1(Total  =  100 

A  Logical  %  B.  Command  %  C.  Ma t hema t i ca 1 


1)  .  Da  t  a  Man  i  pu  1  ,i  t  i  on 


o, 

Art 


E .  Da t  a  Dec  1  a  r  a  t  l on 


■V* 


8.7  Special  Display  Requ  1  rement  s  : 

Simple  I  nput /Output  User  Friendly 

Interactive  _  Complex  Requi rement s/Severe  Impact 

8.8  Languages  Used  as  a  Percent  of  Item  8.1 (Total  =  100%): 


A.  Language: 

B.  Language: 

C.  Language: 

1) .  Language: 

8.9  Reusable  Code  From  Similar  Projects 


Percentage : 
Percentage : 
Percentage : 
Percentage : 


% 

X 

% 

% 


Project 


%  of  Modification  Required 
/»  of  DSLOC _  Design  _  Code  _  Integration 


X 

o, 

A 

% 


X 

.  % 

% 

% 


9.  Documentation 


Document  title*  P  of  Pages  F.st'd  or  Act  '  1 

CPC  I  Development  Specification 

CPC1  Product  Specification 

Test  Plan 

! e  s t  Procedu re  s 

Test  Report 

User' s  Manna  I 

( *pe  ra  tor  s  Manna  1 

I  >t  he  r  : 

Ot  he r  : 

Ot  her 


It  -  1  <1 


V-W.C-C 
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COMPUTER  PROGRAM  CONFIGURATION  ITEM 
SUMMARY  DATA  FORM  INSTRUCTIONS 


Item  _1  :  Enter  the  name  of  the  CPC  I  for  which  this 
form  is  being  prepared. 

I  Lem  2:  Enter  a  brief  description  of  the  major  func¬ 

tions  performed  by  this  CPC I . 

I tem _ 3.1:  Enter  the  original  schedule  date  for  each 

applicable  milestone  (enter  N/A  if  a  milestone  is  not  applic¬ 
able).  It  the  milestones  are  not  established  for  this  project, 
use  the  schedule  data  for  equivalent  activities  to  complete 
this  item.  Formal  milestone  definitions  are  included  in  Attach¬ 
ment  A.  Unless  otherwise  indicated,  the  date  should  reflect 
the  activity  completion  date .  Where  available,  enter  the  actual 
date  of  completion  for  the  milestone;  for  ongoing  efforts, 
enter  the  current  estimate  for  completion  of  the*  milestone. 

Item  3.2:  Indicate  with  an  X  the  degree  of  schedule 

acceleration  or  stretchout  that  the  original  schedule  dates  in 
Item  3.1  represent  relative  to  the  normal  time  required  to 
develop  this  CPC  I.  For  example,  if  the  specified  schedule  is 
24  months  and  the  normal  development  time  is  est  i  mated  at 
10  months,  the  schedule  accelerat  ion/st  rctchout  is  <30°/,. 

Item  4.1:  For  ea< h  of  the  five  areas  listed,  indi¬ 

cate  the  average  experience  at  the  start  of  this  project  of 
the  development  personnel  working  on  this  CpC 1  .  Item  4 .  1 A 
refers  to  experience  with  <  >  t  lie  i  projects  having  similar  func¬ 
tions  and  interfaces.  Item  4.1R  refers  to  experience  with  the 
develop  me  n  t  tools  and  methods  used  on  this  CPC  I  .  Item  4  .  1  C 
refers  to  experiem  e  with  tie  pr  op.ramm  i  ng  I  anguages  used  on 
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this  CPCI.  Item  A. ID  refers  to  experience  with  the  development 
and  target  computer  hardware,  operating  systems  and  architecture 
Item  A. IE  refers  to  experience  with  the  support  software  and 
automated  development  tools,  e.g.  ,  program  design  languages, 
debuggers,  etc.,  used  in  the  development  of  this  CPCI. 

Item  A  .  2  :  Indicate  with  an  X  the  capability  of  the 

analysts  and  programmers  who  are  working  on  this  CPCI  in  terms 
of  percentiles  with  respect  to  the  overall  industry  population 
of  programme rs/des i gne rs .  This  rating  should  be  based  on  apti¬ 
tude  for  programm i ng/des i gn i ng  software,  efficiency  and  thorough 
ness,  and  ability  to  communicate  and  cooperate.  This  rating 
should  reflect  the  capability  of  the  personnel  as  a  team  rather 
t  han  ind i vi dua 1 s . 

I t  em  A. 3:  Indicate  as  a  percentage  the  degree  to 

which  manpower  loading  levels  are  constrained  by  personnel 
availability  or  budget  limitations.  An  entry  of  85%  indicates 
that  the  attainable  manpower  loading  was  15%  less  *  nan  the 
required  level.  If  there  are  no  manloading  constraints, 
enter  100%. 

I  t  em  A  :  For  completed  developments  enter  the  peak 

manioading  level,  i.e. ,  the  largest  number  of  software  engi¬ 
neers  and  programmers  at  a  point  in  time,  attained  during  the 
development  of  this  CPCI .  For  ongoing  developments  enter  the 
current  estimate  of  the  maximum  manloading  required. 

I  t  em  5:  Indicate  with  an  X  the  level  of  reliability 

• ■  -qu i red  for  this  CPCI  using  the  following  impact  criteria: 

V<-ry  Low  =  The  impact  of  a  software  1  a  l  1  u  re 
is  simply  t  lie  inconvenience  caused  by  the 
requirement  io  fix  the  software.  'lypical 
examples  are  a  demonstration  prototype  of  a 
voice  typewriter  or  an  early  feasibility  phase 
software  sirnulat  ion  model  . 


Low  =  The  effect  of  the  failure  is  a  small, 
easily  recoverable  loss  to  the  users.  Typical 
examples  are  a  long  range  planning  model  or  a 
climate  forecasting  system. 

Nominal  =  The  effect  of  software  failure  is  a 
moderate  loss  to  users,  but  a  situation  from 
which  one  can  recover  without  extreme  penalty. 

Typical  examples  are  management  information 
systems  or  inventory  control  systems. 

High  =  The  effect  of  the  software  failure  can 
be  a  major  financial  loss  or  a  massive  human 
inconvenience.  Typical  examples  are  banking 
systems  and  electric  power  d i s t r i bu t i on  systems. 

Very  High  =  The  effect  of  software  failure  can 
be  the  loss  of  human  life.  Examples  are  mili¬ 
tary  command  and  control  systems  or  nuclear 
reactor  control  systems. 

Item  6:  Indicate  with  an  X  the  level  of  complexity 

of  this  CPCI  using  the  criteria  in  Table  B-l  by  matching  the 
characteristics  for  each  type  of  processing  performed  by  this 
CPC  I  . 


Item  7 :  Indicate  with  an  X  the  degree  of  precision 

in  the  development  specification  using  the  following  criteria: 

Very  precise  -  No  additional  analysis  is 
needed  to  develop  detail  design 

Precise  =  Only  minor  details  must  be  worked 
out  t  cj  develop  detail  design 

Imprecise  =  Significant  additional  analysis 
is  required  to  develop  detail  design. 

Item  8.1  :  Enter  the  total  deliverable  lines  of  sourct 
code  for  this  CPCI.  Do  not  include  lines  which  are  entirely 
documentation,  such  as,  comments  or  source  instructions  from 
unmodified  utility  software.  This  line  count  should  include 
job  control  language  instructions,  format  statements,  and  data 
declarations  as  well  as  logic  control  instructions. 
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Ncjin  i  ria 


TABLE  B-l 

SOFTWARE  COMPLEXITY  CRITERIA 


TYPE 
RATING  ! 


CONTROL 

OPERATIONS 


Sequenced  code 
with  a  few 
non-nested  SP 
operators:  DOs 
CASEs , IFTHEN- 
ELSEs .  Simple 
pred i ca  tes 

St  ra i ght  f orwa rd 
nesting  of  SP 
operators . 
Mostly  single 
predicates 


Most  1 y  s imp  1 e 
nest i ng .  Some 
i nte  rmodu 1 e 
cont  ro 1 . 

Dec i s i on 
t  a  h  1  e  s 


Highly  nested 
SP  operators 
with  many  com¬ 
pound  pred i - 
cates.  Queue 
and  stack  con¬ 
trol  . 

Cons i de  rah  1 e 
i nt  e rmodu 1 e 
c  ont  ro 1 


COMPUTATIONAL 

OPERATIONS 


Evaluation  of 
simple  ex¬ 
press  ions ,  e . g . 
A=B+C*(D-E) 


Evaluation  of 
moderate  level 
express  ions 
e.g.,  D=SQRT 
( .  "A"C ) 


Use  of  standard 
math  and  sta¬ 
tistical  rou- 
t  i  nes .  Bas l c 
matrix  and 
vector  oper- 


Bas  i  c  numer i ca 1 
analysis:  multi 
variate 
i  nterpo 1  a t i on , 
ordinary  dif¬ 
ferential 
equa  t  i  oris  . 

Basic  trunca- 
t i on  roundot f 
conce  rns 


DEVICE-DEPENDENT 

OPERATIONS 


Simple  read, 
j  write  state¬ 
ments  with 
:  simple  formats 


.  No  cognizance 
needed  of  par¬ 
ticular  proc¬ 
essor  or  I/O 
|  device  charac- 
!  ter i s t i cs .  1/0 
:  done  at  GET/ 
PUT  level,  no 
cognizance  of 
overlap 

I/O  processing 
includes  de- 
.  vice  selection 
1  Status  check¬ 
ing  and  error 
process i ng 


Operations  at 
physical  I/O 
1  eve  1 ( phys  l  ca  1 
storage  ad¬ 
dress  t  rails  1  a  - 
t  l  ons ,  seeks , 
reads ,  etc.) 
Optimized  I/O 
ove  r 1 ap 


DATA 

MANAGEMENT 
OPERATIONS 
1  ~ 

i  Simple  arrays 
|  in  main  memory 


Single  file 
subsett ing 
with  no  data 
st  ruct ure 
changes  ,  no 
data  edits, 
no  inter¬ 
mediate  files 


Multiple  input 
and  single  f i 1 e 
output.  Si mp 1 e 
structural 
changes  simple 
edits 


Special  purpose 
subrout i nes 
<i  c  t  i  va  ted  by 
data  stream 
contents. 
Complex  data 
rest  rue Luring 
at  record 
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TYPE 

CONTROL 

T 

COMPUTATIONAL 

DEVICE-DEPENDENT 

DATA 

MANAGEMENT 

OPERATIONS 

RATING 

OPERATIONS 

OPERATIONS 

OPERATIONS 

Verv 

Reentrant  and 

Difficult  but 

Routines  for 

Genera  1 i zed 

High 

recursive  cod¬ 

st  ruct u  red 

interrupt 

pa  rameter- 

ing.  Fixed- 

nume r l ca 1 

d i agnos i s , 

d  r i ven  file 

priority 

ana  lysis: 

se  rv i c i ng , 

st  ructur i ng 

1 nte  rrupt 

nea r-s ingu 1  a  r 

masking . 

rout i ne .  File 

hand  ling 

ma  t  r  i  x  equa  - 

Comnnin  k  at  ion 

hu i 1 d i ng  ,  com¬ 

t  i  ons ,  partial 

1  l ne  hand  ling 

mand  processing, 

different i a  1 

sea  rch 

equa  t i ons 

opt i mi za  t i on 

Ext  ra 

Multiple  re¬ 

Difficult  and 

Device  timing- 

Highly  coupled, 

High 

source  sched¬ 

unst  ruct  tired 

dependent 

dynamic  rela- 

uling  with 

nume  r i ca 1 

coding,  micro¬ 

t i ona  1  st  rut  - 

dynant  i  ca  1  1  y 

ana  1 ys 1 s : 

prog  rammed 

Lures . 

chang i ng 

highly  accu¬ 

operat i ons 

Natura 1 

pr i ori t i es  . 

rate  analysis 

language  data 

M i c rocode 

of  noisy 

management 

level  control 

stochast ic  data 

i 

Item  8.2:  Enter  the  total  lines  of  source  code  docu¬ 
mentation  delivered  with  the  source  code  for  this  CFCI . 


Item  8 . 3 :  Enter  the  size  of  the  oat  a  base  to  be  devel¬ 
oped  with  this  CPC1  and  delivered  as  part  of  the  system  in 
bytes. 

Item  8 . h :  Enter  the  percent  of  the  deliverable  lines 
of  source  code  that  performs  each  of  the  categories  of  opera¬ 
tion  defined  below: 


Data  Storage  and  Retrieval  -  Operation  of 
data  storage  devices,  data  base  management, 
secondary  storage  handling,  data  blocking  and 
deblocking,  hashing  techniques.  Primarily 
hardware  oriented  code. 

On-line  Communications  -  Including  machine-to- 
machine  communications  with  queuing  allowed. 
Timing  restrictions  not  as  severe  as  with 
real-time  command  and  control. 

Real-time  Command  and  Control  -  Machine-to- 
machine  commun i ca t i ons  under  tight  timing 
constraints.  Queuing  not  practicable.  Heavy 
hardware  interface.  Strict  protocol  require¬ 
ment  s  . 

Interactive  Operations  -  Real-time  man/machine 
interfaces.  Human  engineering  considerations 
and  error  protection  are  very  important. 

Mathemat  ical  Ope  rat  ions  -  Rout  ine  mathemat  ical 
applications  with  no  overriding  constraints. 

String  Manipulation  -  Routine  applications 
with  no  overriding  constraints.  Not  oriented 
towards  mathematics.  Typified  by  language 
compilers,  sorting,  formatting,  buffer  manip¬ 
ulation,  etc. 

Operating  Systems  -  Task  management.  Memory 
management  .  Heavy  hardware1  interface.  Many 
interactions.  High  reliability  and  strict 
t i ming  requirements. 


Other  -  Specifically  identify  any  unique  opera¬ 
tions  not  included  in  the  above  categories. 


Item  8.5:  Indicate  with  an  X  the  response  mode  re¬ 

quired  in  the  operational  system  using  the  following  guideline 


Real-time  -  The  software  must  complete  proc¬ 
essing  in  response  to  an  event  prior  to  the 
occurrence  of  the  next  ..vent.  Arrival  of  the 
data  and  the  occurrence  of  events  is  not  under 
the  control  of  the  software  and  extra  effort 
in  the  design,  test  and  implementation  of  the 
software  is  required  to  satisfy  time  and  proc¬ 
essing  requirements. 

On-line  -  Software  in  this  category  must  re¬ 
spond  within  a  human  compatible  time  frame, 
usually  within  a  few  seconds.  Also  requires 
additional  development  effort,  but  not  the 
extra  level  required  for  real-time  software. 

Time-constrained  -  Software  in  this  category 
must  complete  processing  within  a  specified 
t  ime  frame  which  is  not  as  restrict  ive  as 
real-time  or  on-line  requirements.  Time  lines 
are  in  the  order  of  minutes  or  hours;  some¬ 
times  a  clock  time  is  specified  for  comple¬ 
tion  of  processing. 

Non- time  Critical  -  There  is  no  time  constraint 
for  completion  of  processing  for  this  category 
of  soft  wa  re  . 


I t em  8.6:  Enter  the  percentage  of  the  delivered  line 

of  source  code  for  this  CPCI  for  each  of  the  statement  types 
listed  using  the  following  guidelines: 


Logical  -  statements  which  control  the  execu¬ 
tion  sequences  in  the  program  and  include 
constructs  such  as  I F-THEN- ELSE ,  IX)  WHILE,  IX) 
UNTIL,  CASE,  CO  TO  <>r  CALL. 
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Command  -  statements  which  direct  the  system 
software  to  perform  specific  functions  or  to 
create  the  environment  required  to  support 
the  software.  These  statements  are  generally 
written  in  a  language  specific  to  the  computer 
hardware . 

Mathematical  -  statements  which  perform  compu¬ 
tations.  This  category  includes  coded  equa¬ 
tions  for  algorithms,  vector  algebra,  modeling, 
index  computation,  etc. 

Data  Manipulation  -  statements  which  perform 
input  and  output,  as  well  as  the  storage, 
movement  and  modification  of  data.  Format 
statements  are  also  included. 

Data  Declaration  -  statements  which  are  non¬ 
executable  and  define  the  characteristics  and 
values  of  the  data  contained  in  the  program. 

1 1  em  8.7:  Indicate  with  an  X  the  level  of  display 
interaction  required  for  the  user  interface  with  this  CPCI 
using  the  following  guidelines: 

Simple  Input/Output  -  No  special  considerations 
or  requirements  implemented  to  enhance  user 
interface. 

User  Friendly  -  Special  display  formatting, 
e.g.  ,  menus,  with  extensive  diagnostic  and 
input  or  processing  error  handling  capabilities. 

Interactive  -  Advanced  features  such  as  light 
pen  or  touch-sensitive  displays  for  user  inter¬ 
face  . 

Complex  Kequ i remen t s/Seve re  Impact  -  Two  di¬ 
mensional  projection  of  solid  figures,  hidden 
line  processing  in  1 i ne -o f - s i gh t  projections, 
earth  projections  from  space  for  weather  pre¬ 
diction  and  resource  mapping  and  integrated 
ci rcui t  layout  . 

Item  8  ^8 :  Identify  the  programming  languages  in  which 

this  CPCI  is  written  and  indicate  the  percentage  of  the  delivered 
source  lines  coded  in  the  language. 
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it.emh._9:  Identify  any  previously  developed  code 

which  has  been  adapted  for  this  CPC I.  Include  the  name  of  the 
project  for  which  the  code  was  developed  and  the  number  of 
delivered  source  lines  of  code  (DSLOC)  which  were  adapted. 
Enter  the  percentage  of  the  original  design  which  was  modified 
and  the  percentage  of  code  which  was  rewritten.  Finally,  indi¬ 
cate  the  approximate  percentage  of  effort  required  to  integrate 
and  test  the  adapted  software  compared  to  the  normal  amount 
required  for  a  new  development  of  a  comparable  size  and  dif¬ 
ficulty. 


1 t em _ 9:  Enter  the  page  count  for  each  document  listed 

and  specify  any  additional  documentation  required  for  this 
CPCI  .  Indicate  with  an  X  in  the  appropriate  column  whether 
the  page  count  is  estimated  or  actual. 

Item  10:  Enter  the  number  of  software  failures,  design 
errors  and  coding  errors  discovered  during  each  of  the  five 
development  phases. 

1 1  em  1_1  :  Describe  any  other  factors  or  characteristics 
of  this  CPC  1  and  the  development  environment  which  affected 
the  number  of  delivered  source  lines  of  code  or  the  resources 
expended  in  the  development  of  this  CPCI.  Identify  any  features 
of  this  development  which  make  it  unique  relative  to  other 
deve 1 opmen t  s . 
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RESOURCE  EXPENDITURE  DATA 

1.  Project  Name:  _ 

2.  Latest  Month  of  Actuals:  _  3.  Units  of  Nanpowe 


RESOURCE  EXPENDITURE  DATA  FORM  INSTRUCTIONS 


This  form  is  designed  to  collect  time-phased  manpower 
data  for  the  software  development  project  at  the  lowest  level 
of  detail  available.  Attach  a  copy  of  the  cost/work  breakdown 
structure  used  to  collect  manpower  data  for  software  activities 
on  this  contract/development  project. 

I  tern  1:  Enter  the  project  name. 

1 t  em  2  :  Enter  the  latest  month  after  contract  award 
or  project  start  for  which  actual  manpower  data  is  available. 
This  number  should  reflect  the  months  after  the  date  for  con¬ 
tract  award  entered  in  Item  6  for  the  contract  award  milestone 
on  the  Software  Development  Project  Summary  Data  Form. 

Item  j:  Enter  the  units  of  measure  used  for  the  man¬ 

power  figures,  that  is,  manhours,  mandays  ,  inanition  ths  or  manyears. 
Indicate  the  number  of  hours  that  the  unit  is  based  on,  if  you 
are  not  entering  manhours. 

1  n  t  lie  top  row  of  the  table  enter  the  name  of  the 
cost /work  breakdown  structure  element  or  computer  program  con- 
f igurat  ion  item  for  which  you  have  manpower  data.  Include  any 
of  the  software  related  work  breakdown  structure  elements  in 
Attachment  B  tor  which  you  have  data.  In  each  of  the  subsequent 
rows  enter  the  manpower  expended  during  the  month  after  contract 
award  indicated  in  the  left  hand  column.  Space  is  provided 
for  up  to  five  years  of  data.  For  ongoing  projects  enter  the 
latest  est  incite  of  resource  requirements  for  those  months  for 
which  actual  data  is  not  available. 
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COMPUTER  PROGRAM  CONFIGURATION  ITEM 
FUNCTION  AND  SIZING  DATA  DETAIL  FORM  INSTRUCTIONS 


Enter  the  name  of  the  CPC1  for  which  the  information 
is  being  furnished  in  the  space  provided.  For  each  module  or 
unit  in  the  CPCI,  identify  computer  component  to  which  it  be¬ 
longs,  the  function  index  number  from  Table  B-2,  the  module 
size  in  delivered  source  lines  of  code  and  number  of  main  mem¬ 
ory  words  occupied,  and  the  programming  language  used.  Like¬ 
wise,  indicate  with  an  X  in  the  appropriate  column  whether  the 
code  is  reused  code  with  little  or  no  modification,  reused  cod< 
with  extensive  modification,  or  entirely  new  code.  If  informa¬ 
tion  is  not  available  at  the  module  or  unit  level,  provide  it 
at  the  next  highest  I  eve  1  available,  i.e.,  CPC  level.  If  the 
CPCI  does  not  use  all  o{  the  intermediate  levels,  leave  the 
unused  column  blank.  In  the  event  that  none  ot  t  lie  functions 
in  Table  B-2  adequately  describes  a  particular  CPCI  element, 
enter  a  brief  statement  of  the  element  function  in  the  appro- 
p  r i a  t e  co 1 umn . 


B-  34 


— *w  - 


v  -  .  -  .  •  .  •  . /  ~  ■/  ~~ 


^  ttimit'"'  *~"r *-’4*-' 


TABLE  B-2 

SOFTWARE  FUNCTION  CATEGORIES 


TYPE 

— 

CATEGORY 

INDEX 

FUNCTION 

Ope  rat lona 1 

Displays 

1  .  1 

1.2 

Avioni cs 

Command,  Control,  Communications 

j 

1.3 

Other 

,  Avionics 

2.  1 

Mission  Planning 

1 

9  ') 

Nav i gat l on 

2.3 

Aircraft  Steering  N  Flight  Control 

2.4 

Sighting,  Designation  X  Location 

Determi nat i on 

l 

2.5 

Weapon  Delivery 

' 

2.6 

Elect  rori  i  c  Conn t e rmeasu  res 

' 

2.7 

Other 

-  -H 

Command , 

Control ,  & 
Commun i cat  ions 

3.  1 

3.2 

Network  Monitoring 

Network  Control  Switching 

1 

3.3 

Sensor  Control 

3.4 

Signal  Processing 

3.5 

Message  Processing 

3 . 6 

Message  Distribution 

3.7 

Message  Logging  &  Retrieval 

3.8 

Data  Reduction 

3.9 

Other 

Execut i ve 

4 .  1 

Computer  Resource  Management 

4 . 2 

Computer  Operator  lutert.ue 

4.3 

Other  Terminal  Operator  lnleit.t  • 

4.4 

Spec  i  a  1  Dev  i  ce  lute:  1  1 i  >■ 

4.5 

other  1  nput  <■  r  '  pi. 

4 . 6 

Er  ror  Hand  1  l  ng  T.n  <  ■  :  .  , 

Re< uve  rv 

4.7 

Mu  1  t  i  (  '  Miip'i  t  •  • 
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TABLE  B-2 

SOFTWARE  FUNCTION  CATEGORIES  (Continued) 


TYPE 

CATEGORY 

INDEX 

FUNCTION 

Operational 

Data  Base 

5.1 

On-Line  Retrieval  &  Output 

(Continued ) 

5.2 

On-Line  Initialization  &  Updating 

5.3 

Other 

T  ra i ning 

6 . 1 

Control  of  Exercise  Sequencing 

6.2 

Operator  Performance  Data  Collection 

6.3 

Other 

On-Line 

7.1 

System  Readiness  Test 

Equipment 
Diagnost i c 

7.2 

Computer  Diagnostic 

7.3 

Memory  Diagnostic 

7. A 

Display  Diagnostic 

7.5 

Swi tch/ I ndi cator  Panel  Diagnostic 

7.6 

1/0  Diagnostic 

i 

7.7 

Mode  Diagnostic 

_ 1 _ 

7 .  B 

Other 

Support 

Ope rat  1 ng 

8.  1 

Computer  Resource  Management 

i  system 

8.2 

Computer  Operator  Interface 

8.3 

Terminal  Operator  Interface 

8.4 

Input  or  Output 

8.5 

Error  Handling/Reconf igurat ion/ 

Recovery 

: 

8.6 

Performance  Monitoring  6  Data 

Col lect ion 

8.7 

Other 

Equipment 

9.  1 

Off-Line  Computer  Diagnostics 

Na  intenance 

_ 

9.2 

Other 
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Type 


Category 


I  ndex 


Function 


Support 

Software 

10.1 

Higher  Order  Language  Compiler 

(Continued) 

Development 

10.2 

Assembler 

10.3 

Debugger 

10.4 

Loader  or  Editor 

_ 

10.5 

Other 

Off-Line 

11.1 

Data  Base  Definition 

Data  Base 

Management 

11.2 

Data  Base  Initialization  or  Updating 

11.3 

Retrieval  &  Output  Formatting 

1 1.4 

Data  Base  Restructuring 

: 

I 

11.5 

Off-Line  Data  Base 

i 

1 1.6 

Other 

Design 

12.1 

Data  Base  Design 

12.2 

Data  Base  Processor  Design 

i 

12.3 

Performance  Simulation 

12.4 

Data  Reduction 

, 

12.5 

Data  Analysis 

' 

12.6 

Other 

Test 

13.1 

Test  Case  Generation 

So  f  t  wa  rp 

13.2 

Test  Case  Data  Recording 

13.3 

Test  Data  Reduction 

1 

13.4 

Test  Analysis 

13.5 

Other 

Utilities 

14.  1 

Media  Conversions 

14.2 

Format  Translation 

14.3 

Sort/Merge 

14.4 

Program  Library  Maintenance 

, 

i 

1 _ 

14.5 

Other 

TABLE  B-2 

SOFTWARE  FUNCTION  CATEGORIES  (Continued) 


TYPE 

CATEGORY 

— 

INDEX 

FUNCTION 

Support 

Of  f-Line 

15.1 

Data  Reduction 

(Continued) 

Training 

15.2 

Training  Analysis 

15.3 

Scenario  Preparation 

... 

15.4 

Other 

Project 

16.1 

Project  Event  Status  Accounting 

Management 

16.2 

Schedule  Maintenance/Project  ion 

16.3 

Financial  Accounting 

16.4 

Software  Cost  Reporting 

16.5 

Hardware  Cost  Reporting 

16.6 

Software  Cost  Prediction 

16.7 

Hardware  Cost  Prediction 

16.8 

Other 

Ha  rdwa  re 

17.  1 

Interfacing  Hardware  Simulations 

Subsystem 

Simulations 

17.2 

Environmental  Simulations 

17.3 

Operator  Action  Simulations 

17.4 

Other 

ATTACHMENT  A 
GLOSSARY 


Application  software  -  software  that  implements  the  operational 
capabilities  of  a  system . 

Assessment  -  a  qualitative  evaluation. 

Compiler  -  a  computer  program  that  accepts  a  source  program 
expressed  in  a  higher  order  language  as  input ,  and  produces 
either  a  machine  code  or  assembly  language  representation  of 
the  source  program  as  output. 

Code  walk-through  -  a  step-by-step,  detailed  examination  of 
source  code  by  a  small  group  of  qualified  personnel.  Some¬ 
times  it  is  referred  to  as  a  peer  review. 

Computer  data  -  basic  elements  of  information  used  by  the 
computer  hardware  in  responding  to  a  computer  program. 

Computer  program  -  a  series  of  instructions  or  statements  in  a 
form  acceptable  to  an  electronic  computer  designed  to  cause 
the  computer  to  execute  an  operation  or  a  series  of  operations. 

Computer  software  -  a  combination  of  associated  programs  and 
computer  program  data  definitions  required  to  enable  the  com¬ 
puter  hardware  to  perform  compu t a t iona 1  or  control  functions. 
Note:  this  definition  includes  firmware. 

Computer  program  component  (CPC)  -  a  design  component  of  the 
computer  program  design  archi tecture  made  up  of  units  and 
modules  implementing  requirements  of  a  computer  program  con¬ 
figuration  item  (CPC1). 

Computer  program  configuration  item  (CPCI)  -  an  aggregation  of 
computer  software  whicn  satisfies  an  end  use  function  and  is 
designated  for  configuration  management. 

Contractor  -  any  organization  under  contract  or  tasking  agree¬ 
ment  with  a  procuring  agency  to  perform  any  part  of  a  software 
development  effort. 

Critical  Design  Review  (CbR)  -  a  review  conducted  for  each  con- 
Fi  gurat ion  item  when  the  detail  design  is  essentially  complete 
to  determine  if  the  detail  design  satisfies  the  requirements 
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established  in  the  specification  and  to  establish  the  exact 
interface  relationships  with  other  parts  of  the  system. 

Defense  system  -  a  system  which  contributes  directly  to  the 
combat  capabi 1 i ty  of  the  Department  of  Defense. 

Demonstration  -  a  qualification  method  which  relies  on  observ¬ 
able  operation  to  establish  that  a  requirement  has  or  has  not 
been  met . 

Design  walk-through  -  a  step-by-step  detailed  examination  of 
the  design  oT  the  software  by  a  small  group  of  qualified 
pe  rsonne 1 . 

Development  basel ine  -  the  initial  approved  technical  documen¬ 
tation  wh i c K  de f i nes  the  con f i gura t i on  of  a  CFC1  during  the 
Full-Scale  Development  Phase  and  which  prescribes  (1)  all  design 
characteristics  of  the  CPCI  and  (2)  the  selected  functional 
characteristics  of  the  CPCI  designated  for  software  performance 
testing.  The  Developmental  Baseline  is  under  the  development 
contractor's  configuration  control. 

Documentation  -  the  eomprehensi ve  written  description  of  com¬ 
puter  software  in  various  formats  and  levels  of  detail  that 
clearly  define  its  content,  composition,  design,  performance, 
test i ng ,  and  use . 

Embedded  computer  sof t ware  -  software  which  executes  on  com¬ 
puters  whlcTi  are  ( 1  )  incorporated  as  integral  parts,  (2)  dedi¬ 
cated  to  or  required  for  the  direct  support  of,  or  (3)  required 
to  upgrade  or  modify  defense  systems. 

Firmware  -  microcode  (software)  which  resides  in  computer  memory 
That  is  not  alterable  by  the  computer  system  during  program 
execut i on . 

Formal  Qualification  Test  (FQT)  -  a  formal  test  conducted  in 
accordance  with  approved  test  plans,  descriptions,  and  proce¬ 
dures  after  a  CPCI  has  been  integrated  to  validate  that  each 
function  of  the  CPCI  satisfies  specified  software  requirements 
and  applicable  interface  requirements. 

Formal  t  est  -  a  test  which  is  conducted  in  accordance  with  test 
procedures  approved  bv  the  procuring  activity,  is  witnessed  by 
an  authorized  representative,  and  is  documented  in  a  test  report 
for  procuring  agency  review. 

Functional  Configuration  Audit  (FCA)  -  the  formal  examination 
of  func t i onaT  characteristics  test  data  for  a  configuration  item 
to  verify  that  the  item  has  achieved  the  performance  specified 
in  its  functional  or  allocated  configuration  identification. 


Higher  order  language  -  a  primarily  machine  independent  lan¬ 
guage  (of  a  higher  order  than  assembly  language)  designed  for 
ease  of  expression  of  a  class  of  problems  or  procedures  by 
humans . 

Inspection  -  a  qual i f ication  method  which  relies  on  visual 
examination  to  establish  that  a  requirement  has  or  has  not 
been  met . 

Measurement  -  quantitative  evaluation. 

Modular  -  a  software  design  characteristic  which  organizes  the 
software  into  limited  aggregates  of  data  and  contiguous  code 
that  perform  complete  functions  and  are,  therefore,  completely 
understandable  by  themselves. 

Module  -  a  discrete,  identifiable  set  of  instructions  which 
are  treated  as  an  entity  by  the  computer's  operating  system, 
and  which  can  be  executed  and  tested  on  a  stand-alone  basis. 
Equivalent  to  a  unit. 

Physical  Configuration  Audit  (PCA)  -  the  formal  examination  of 
tKe  nas-coded"  configuration  of  a  CPC1  against  its  technical 
documentation  in  order  to  establish  the  initial  product  con¬ 
figuration  identification. 

Precompiler  -  a  computer  program  which  converts  programming 
statements  which  are  unacceptable  to  the  compiler  into  state¬ 
ments  with  acceptable  syntax. 

Preliminary  Design  Review  (PDR)  -  a  review  prior  to  the  start 
of  the  detail  design  process  to  evaluate  progress  and  technical 
adequacy  of  the  selected  design  approach,  to  determine  the 
design  compatibility  with  the  performance  requirements  of  the 
CPCI  development  specification,  and  to  establish  the  existence 
and  compatibility  of  the  interfaces  between  the  configuration 
item  and  other  elements  of  the  system. 

Preliminary  Qual i f icat ion  Test  (PQT)  -  a  test  conducted  during 
the  integration  of  a  CPCl  to  evaluate  the  performance  of  those 
CPCI  functions  which  are  critical,  as  determined  by  t ime-cri t i cal 
or  per formance-cri tical  requirements.  A  PQT  may  be  either 
formal  or  informal. 

Product  baseline  -  the  final  approved  technical  documentation 
which  defines  the  configuration  of  a  CPCI  at  the  completion  of 
software  performance  testing  at  the  point  where  the  CPCI  is 
integrated  into  the  system.  The  product  baseline  includes 
final  versions  of  all  specifications  and  documents  from  pre¬ 
ceding  baselines. 
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Program  design  language  -  a  design  tool  used  to  facilitate  the 
translation  of  system  functional  requirements  into  the  elements 
of  a  program  design  hierarchy. 


Program  support  library  -  a  repository  for  programs  and  data 
used  to  faci 1 i tate  the  orderly  development  of  software.  The 
repository  provides  two  fundamental  capabilities:  (1)  programs 
and  data  are  stored  in  machine  readable  form  for  computer  opera¬ 
tion  and  the  identical  information  is  stored  in  hard  copy  form 
for  human  comprehension,  and  (2)  the  repository  contains  all 
management  data  pertinent  to  the  software  development  project. 

Software  development  -  the  engineering  process  and  effort  that 
results  in  software,  encompassing  the  span  of  time  from  initia¬ 
tion  of  the  contracted  effort  through  delivery  to  and  acceptance 
by  the  procuring  agency. 

Software  error  -  an  occurrence,  or  lack  thereof,  during  the 
execution  of  a  program  and  attributable  to  the  software  that 
prevents  satisfaction  of  the  specified  software  requirements, 
fails  to  perform  as  designed,  or  performs  a  function  not  re¬ 
quired  and  not  desired. 

Support  software  -  all  software  used  to  aid  the  development, 
testing  ancTsupport  of  applications,  systems,  test  and  mainte¬ 
nance,  and  trainer  software. 

Systems  software  -  software  that  is  used  to  maximize  the  use 
of  computer  resources  at  the  time  of  use.  Operating  systems, 
executives,  and  data  base  management  systems  are  examples  of 
this  type  of  software. 

System  Design  Review  (SDR)  -  a  review  conducted  when  the  def- 
i  in  i  t  i  oh  e  f  fort  has  proceeded  to  the  point  where  system  require¬ 
ments  and  the  design  approach  are  more  precisely  defined.  The 
review  ensures  that  there  is  a  technical  understanding  between 
the  contractor  and  the  procuring  agency  on  the  system  segments 
identified  in  the  system  specification  and  the  configuration 
items  identified  in  the  Cl  performance  specifications. 

Sys  t  em  Requ i remen  t  s  Review  (SRR)  -  a  review  conducted  when  a 
signTTicant  port  ion  of  the  system  functional  requirements  have 
been  established  to  determine  the  adequacy  of  the  contractor's 
efforts  in  defining  system  requirements. 

Test  -  a  qualification  method  which  relies  on  operation  in  an 
actual  or  simulated  environment  and  subsequent  analysis  of 
data  obtained  during  operation  to  establish  that  a  requirement 
has  or  has  not  been  met . 


Top-down  design  -  a  design  method  in  which  operations  are  de¬ 
fined  in  a  hierarchical  manner  from  the  more  general  to  the 
more  precise.  Top-level  operations  are  defined  in  relatively 
general  terms.  Operations  on  all  levels  except  the  bottom  one 
are  defined  more  precisely  in  terms  of  operations  on  the  level 
immediately  below. 

Top-down  programming  -  the  implementation  of  code  and  data  in 
a  sequence  which  proceeds  from  the  top  level  of  design  to  the 
bottom  level,  continuously  exercising  the  actual  interfaces 
between  program  elements. 

Unit  -  the  lowest  level  logical  entity  specified  in  the  detailed 
design  which  completely  describes  a  non-divi sible  function  in 
sufficient  detail  to  allow  implementing  code  to  be  produced, 
assembled  or  compiled,  and  tested  independently  of  other  units. 
Equivalent  to  a  module. 

Validation  of  computer  software  -  the  evaluation,  integration, 
and  t e s t  ac  ti v! t ies  carried  out  at  the  system  level  to  ensure 
that  the  finally  developed  system  satisfies  the  user's  require¬ 
ments  set  down  as  performance  and  design  criteria  for  the  system 

uter  software  -  the  interactive  process  of 
the  product  of  each  step  of  the  software 
fulfills  all  requi remen t s  levied  by  the 


Virtual  machine  -  the  complex  of  hardware  and  software  that  the 
sof tware  being  developed  calls  upon  to  accomplish  its  tasks. 


Verification  of  comp 
determining  whether 
development  process 
previous  step. 


ATTACHMENT  B 

SOFTWARE  DEVELOPMENT  PROJECT  WORK  BREAKDOWN  STRUCTURE 


The  following  is  an  illustration  of  a  recommended 
work  breakdown  structure  for  a  project  with  both  hardware  and 
software  elements. 


LEVEL  I _ LEVEL  2 _ LEVEL  3 _ LEVEL  4 _ 

Defense  System 

Prime  Mission  Equipment 

Integration  and  Assembly 
Hardware  Subsystem  or  End  Item  1 


Hardware  Subsystem  or  End  Item  n 
Software  Subsystem  1 

Subsystem  Analysis  &  Design 
Subsystem  Integration  &  Test 
Computer  Program  Configuration  Item  1 


Computer  Program  Configuration  item  n 


Software  Subsystem  n 


Level  1 


Level  2 


Level  3 


Level  A 


Support  Software 

Computer  Program  Configuration  Item  1 


Computer  Program  Configuration  Item  n 


Training 


Equipment 
Services 
Faci 1 i t ies 

Peculiar  Support  Equipment 

Organizational 

Intermediate 

Depot 

System  Test  &  Evaluation 

Development  Test  &  Evaluation 
Operational  Test  &  Evaluation 
Mockups 

Test  &  Evaluation  Support 
Test  Facilities 
System/Program  Management 

Systems  Engineering 
Project  Management 

Data 

Technical  Publications 
Engineering  Data 
Management  Data 
Support  Data 
Data  Depository 


Level  1 


Level  2 


Level  3 


Level  U 


Operational/Site  Activation 

Contractor  Technical  Support 
Site 

Construction 

Site/Ship/Vehicle  Conversion 
System  Assembly  Installation  & 

Checkout  on  Site 
Common  Support  Equipment 

Organizational 

Intermediate 

Depot 

Industrial  Facilities 

Construct  ion/ Conversion/ Expans  ion 
Equipment  Acquisition  or  Modernization 
Maintenance 

Initial  Spares  &  Initial  Repair  Parts 
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